Ember's Direction re: "Reactive" Programming

Design
1

I’ve been a software engineer / technical leader for about 15 years now, and I have no interest in the pedantic “tabs vs. spaces” type of holy wars that I see rage all the time.

I’ve recently been entertaining the notion of taking a job building and being a technical mentor on a team building a non-trivial web app with ember that seems well-suited to Ember’s use case.

I wanted to quickly drop in to ask for some input. I just watched the Fireside Chat from Ember Conf (a few days back) with @wycats and @ef4 and I felt that the conversation missed some important nuance in the lessons that I’ve learned from building substantial UIs in React over the past 7 years.

Garbage collection / cleanup were a plague to me and my team I was leading in 2013~ish while we were building a large Backbone / Marionette web app. The “hard MVC” patterns that were established there create these silos that things have to fit into categorically. “Service” “Factory” “Model” “View” “Controller” are a few. I saw unreasonable amounts of time wasted arguing about which silo a given “thing” should be put into when it straddled the line between a “service” vs. “factory” and similar debates. React doesn’t care about your file structure or silos. Obviously you need organization / structure in order to have a successful project, but when things don’t need to extend some JS base class, they can just do exactly what they are supposed to, and if you don’t know what “shelf” they belong on, you can just put them in the mis “utils” directory. Native ES5 modules / import statements allow things to depend on each other without undue formality in their contract and no need to compare their relative match to the base class they extend from. Something useful in the React ecosystem has been functional programming, when you add the needed evolution of React hooks / context so that you’re not constantly having to blindly spread props down or bloat your intermediary components with irrelevant passthrough props, things get much better. Along similar lines (and I realize this is an implementation choice that is not applicable to every project) Apollo and GraphQL are powerful in that ecosystem for getting only the data you need. Apollo acts as middleware and de-dupes requests so that each component can call useQuery for itself and not have to worry about a second component in a highly dynamic screen potentially duplicating the data request (a common anti-pattern in REST sans framework, which I think Ember solves via the Factory / Service pattern).

On the subject of state mutation, @wycats mentioned that Starbeam lets you look at the output but React doesn’t. This feels like a misunderstanding, and is the reason Redux is commonly paired with React. It gives you time-traveling state mutation history and the ability to very concisely bisect a problem in a step-by-step debug step through mode (e.g. Chrome Debugger js breakpoints).

All that to say, some lessons I feel could be useful for Ember to learn from the principles of React (and don’t get me wrong, the ecosystem has no shortage of frivolity, and best practices, standards, and incremental change are appealing philosophically after betting on a project and then watching it die!)

  • Functional programming + Redux is powerful, why couldn’t Redux be added under the hood to as a source for ember-data via built-in mechanism to then give time-traveling debugging (immense time savings and developer productivity boosts in my experience with large applications) and avoid re-inventing the wheel?

  • Garbage collection is almost never a problem in React because of functional paradigms and the built-in garbage collection in the framework. Are the Ember lifecycle hooks like isDestroying / willDestroy really beneficial? @wycats mentioned the need to unsubscribe from websockets. In React I’ve not needed to unsubscribe, I can just kill the DOM node and the unsubscription is implicit. Can someone explain this use case to me?

There are more things I could ask, but I’ll start with these questions, mainly because further conclusions I draw might be based on a flawed assumption before I fully understand the reasons for some of these decisions I find confusing. I’m just worried that there’s a lack of real collaboration with those familiar with React and it’s real-world use cases / benefits.

2 Likes
2

@brianfeister thanks for the thoughtful engagement and questions, I think we’d all like to see more of this sort of dialogue between ideas/ecosystems and better understand the similarities and differences in each model.

Preface: I’m just one individual and honestly not all that knowledgeable so take this with a grain of salt, and hopefully others can weigh in too.

First your two questions:

why couldn’t Redux be added under the hood to as a source for ember-data

First off, and maybe you’re aware of this already, Ember does have a redux implementation but obviously it’s separate from Ember Data. Could ember-data use redux under the hood? In theory yes (and this is where my knowledge falls short) but I imagine it would be quite a dramatic overhaul. Perhaps something like that would be more possible after Ember 4.0 and as the internals of Ember Data are cleaned up and refactored more away from the old reactivity model. Using a more flux-y system to back ember data could be really cool but part of me wonders if effort would be better spent continuing to de-emphasize Ember Data’s role as “this is the way to do data with Ember” to “this is one of many options”. This has already changed a great deal and there’s a good Apollo client etc and there has been work done in the guides to give a more even representation of options.

I guess my questions about the idea of backing ember-data with redux would be along the lines of:

  • would it add boilerplate or friction? In my (admittedly limited) experience with Redux is very heavy on boilerplate and takes a lot of effort to use
  • if no and it could be implemented in the guts without “leaking out” everywhere are there any irreconcilable differences in the programming models
  • if it’s buried in ember-data would there be any way to use it for non-Ember-Data state?

Sorry this is probably really unsatisfactory… in summary I don’t really know what I’m talking about but I get the value proposition, I think the question is how much effort would it be and would it really be worth it?

Are the Ember lifecycle hooks like isDestroying / willDestroy really beneficial?

Sure, I guess… in my experience they’re rarely necessary. Most things that need to be cleaned up are better captured in modifiers (and correct me if I’m wrong but useEffect/useLayoutEffect have similar cleanup APIs) or concurrency tasks (which are cancelable and cleaned up by themselves). Lifecycle hooks in general have been deemphasized and largely removed in the Octane model and in my experience in a number of large long-lived Ember apps destruction and it’s just not that big a deal.

Now a few other random comments…

I’m just worried that there’s a lack of real collaboration with those familiar with React and it’s real-world use cases / benefits

I think this concern is very valid and at the same time not so much. I do think we should all actively engage in the React ecosystem more. The more the better right? That’s one of the reasons I feel like conversations like this are great. We can all learn a ton from each other.

That said I think the Ember community does look at React a lot and has since React was first introduced. Certainly much more so than the other way around (which makes sense, if nothing else from a numbers/marketing perspective). A number of Ember’s biggest paradigm shifts and programming model changes have been either influenced by React or at least been taking a sideways glance at React while making a different choice (pre-compiled templates vs JSX, auto-tracking as a reactivity model, etc). Some could argue we’ve straight-up stolen some things from React. A number of our more prolific contributors in recent years have come from a React background or have significant experience with the React tools and concepts. One of the things that really hurt this in the past was how “other” our build system was but now that we’re reaching parity with Embroider and we’ll be able to engage much more fully in the greater javascript ecosystem without friction we’re at a tipping point and I think there will be a LOT MORE engagement with the greater javascript world.

I think with the Polaris roadmap (much of which is already implemented or mostly there) Ember is going to look at lot more like Vue/React and is going to be a thinner glue layer than it ever was in the past, and it will be easier to use the same tools and concepts, and also to compare/contrast where the frameworks differ.

So anyway… I think it’s a valid concern in that we can always use more cross-pollination and inspiration and even stealing. But I don’t think it’s fair to say that we don’t do it much. I think the thing that’s hampered us most is our historical isolation from the rest of the js ecosystem making it much more difficult to leverage tools and to get from/give back to the “outer world”.

React doesn’t care about your file structure or silos

:100: and this is great, but it cuts both ways.

IMHO Ember strikes a nice balance here (YMMV of course). Ember is convention-driven and opinionated and that helps to make some of these decisions for you, and it gives all Ember apps and addons a similar look/feel. Anytime I drop into an addon or app that I’ve never seen I can find things because I know where to look. There are some very legitimate gripes with our current filesystem layout but in my experience it’s just not that big of a deal in Ember apps. Across several large apps and teams I’ve never witnessed any significant debates about where to put things or how to organize them.

Ember also has more “silos” as you call them. This cuts both ways too. Yes, it’s more to learn, yes, occasionally you have to argue about which silo to put something in (although this is also fairly rare in my experience). The tradeoff is that you don’t end up with some of the awkwardness of “everything is a component” or “everything is a function”. You’re not forced to use components as your sole “unit of reactivity”, you don’t end up needing so many types of components like Providers etc.

Anyway… really appreciate these sorts of questions and explorations and really hoping some others will weigh in too so you’re not just stuck with my crazy rambling :stuck_out_tongue_closed_eyes: And of course if you have other questions or topics to discuss would love to hear them!

3

Thanks for the dialogue and taking so much time to answer @dknutsen! :star_struck:

One thing I forgot to say before is that it will be a great story about how powerful Ember is if I’m able to onboard and have impact in a relatively short time. At some level, familiarity with javascript is at play, but still, there is non-trivial ramp up friction with React projects because there is lack of convention around certain tooling that is pretty standard and doesn’t need to be different.

This is good, I think the more I look at it, the more I feel that Ember Data (which feels like Rails in JavaScript) is really the thing that’s bothering me perhaps above all else. The other conventions are fairly loosely coupled and there is some interoperability, but I just don’t see anyone out there talking about Ember’s Redux or Apollo implementations. Are there any places I could go to see non-trivial implementation examples along these lines?

And yes, I agree with you that making redux pluggable under the hood of Ember Data is more squeeze than juice. I guess with the lack of notable examples of other data source libs, I’m just flying a bit blind trying to understand the context better.

This is helpful, thank you. To answer your question – React hooks do return functions usable to call in teardown on the off-chance it’s necessary, but in my experience, I think I’ve only really been forced to handle teardown logic like that once after quite a lot time spent with React. I found myself concerned that the cleanup lifecycle hooks in Ember were a prevalent gotcha due to Ember Data being persistence and needing babysitting in certain teardown contexts. But this thought was purely hypothetical.

This is good to hear, and I do get that sentiment from what I’ve been studying up on. Embroider, in particular being a key milestone in catching up with the interoperability that we see with other JS libs. Of course, JSX and DOM output being the exception that will always diverge.

Yes, and getting native ES6 module support is an absolute necessity in order for that cross-pollination, so yes, bright days ahead for Ember in that regard :sun_with_face:

Something that happens in React is that you start off grouping everything by component, which ends up being a really helpful way of thinking about UI, and with Redux acting as the central place where schema drives everything top down (and most state doesn’t go in Redux, fwiw, they go in local state via useState and only shared data goes in Redux) you hang your nodes (conceptually) off of the main app state tree. What works well is that you start with a service that only gets used by one component, and then when a second component wants to use it, you can just import from the lateral place in the application. When a third consumes it, you decide what it’s “silo” is and either move it there or (very rarely) create that directory. These created directories are the concepts that are central to that application and also unique to it in a good way that “Factory” as a type can’t account for because they’re too specific to your application and that’s a good thing.

FYI useContext allows you to store create Stores which don’t need to be passed around, you just mount them at the app root, the same way that extending the EmberData class automagically causes that item to persist in Ember. So Providers are not so much a weird “is this data or is this DOM” thing in a more modern React app.

Anyway, thanks very much for the conversation, this is super helpful! :bowing_man:

4

I found myself concerned that the cleanup lifecycle hooks in Ember were a prevalent gotcha due to Ember Data being persistence and needing babysitting in certain teardown contexts. But this thought was purely hypothetical

I can’t think of any times I’ve seen Ember Data need babysitting or cleanup per se. That sort of thing is usually related to side-effects or 3rd party libs being captured in a component. Ember Data persists state via its store, which is an Ember service (e.g. a singleton object that has a similar lifecycle to the whole app) so teardown doesn’t usually relate to Ember Data unless you’re writing very irresponsible async code.

This is good, I think the more I look at it…

Yeah that’s fair and for a while Ember Data was just seen as the defacto way of doing data fetching to the detriment of both Ember Data and Ember itself. I’ve seen people try to use Ember Data for things that just don’t fit. That said Ember Data is extremely powerful for the right use cases (minus time travel debugging of course :grin: ). I’d still reach for it by default for any CRUD style ops using a RESTful API. It’s extremely flexible and requires very little boilerplate around caching, requests, serialization, etc.

I think it will just take some time to change the status quo. One step has been not including ember-data in the default blueprint. The guides are much more careful now about presenting Ember Data as being one of many options (though I think we could still take that further).

There are also a number of alternative data libraries:

  • ember-m3: lighter model layer for Ember Data with “dynamic schema”
  • glimmer-apollo - an apollo integration for use with Ember or Glimmer
  • ember-apollo-client - an alternative apollo client integration
  • ember-orbit - an orbit.js integration
  • ember-firebase-service - lightweight service wrapper for firebase, there’s also an ember data adapter that sits on top of this

etc…

Are there any places I could go to see non-trivial implementation examples along these lines?

Unfortunately I’m not sure offhand, but I think that’s largely the problem of “most non-trivial implementations of anything are private”. That said I know there are some people using ember+apollo in prod for larger apps I’m just ignorant of who. Most of the discussion I’ve seen around it is in the Ember discord, so if you want to look more into it I’m sure there are people who would be eager to share more.

Switching gears a little bit…

One thing worth mentioning is that while most of the discussion comparing frameworks seems to focus on technical details, and while Ember (and probably any framework really) has both some shining strengths and some glaring shortcomings in that domain, some of the biggest selling points (imho) are more intangible.

Community consensus driven: Ember core is less “owned” or driven by any one company (facebook, google, etc) and has (i think, at least) been a great example of community organization via mechanisms like the core team, RFC process, editions, release channels, and the commitment to “stability without stagnation”. Convention over config isn’t for everyone, and certainly has some shortcomings (see Ember Data’s chokehold on the data lib choices), but it seems to be a much less fragmented ecosystem. The community is also very positive and supportive and I think that’s worth something.

“Stability without stagnation”: mentioned above but this is a big one. I’ve worked on a number of very large and very long lived apps now that have been ancient by javascript standards but because of Ember’s philosophical commitment to evolve in a developer-friendly way they remained modern and maintainable even with extremely lean development teams. This was possible in a way that few other frameworks and toolsets could really match (ime), because the framework development is carefully driven and the community tends to seek loose consensus on best practices and tools it’s pretty easy to keep current and fresh without having to rewrite the app every year or two.

I could probably drone on about more but I’ll cut myself off there for now :grin:

1 Like
5

Really appreciate all of the discussion @dknutsen, and I agree. While it’s tempting to label some of the innovations as being a bit “behind the curve” in delivery, the ability to keep apps moving forward without extremely painful migration processes between versions is a huge win for productivity. Being very opinionated also means that it should be more feasible to create codemod tools that make migration somewhat automatic if the stars align.

I’m definitely excited about the prospect of working on Ember, something I wasn’t feeling not so long ago when I first turned my attention toward considering a job working with it. I think if a sufficiently experienced engineer can pick it up, work within the conventions and be productive quickly, that would be a huge success story for the architecture and principles espoused by Ember. And I feel pretty good about that being the case.

2 Likes