Ember Data Sycn - LocalStorage + REST + RealTime + Online/Offline Support

Cross-posting from StackOverflow since this lends itself more to discussion than a right/wrong answer.

We have a combination of requirements in terms o data access.

  • Pre-load some reference data.
  • We need reference data to survive browser restarts instead of just living in memory to avoid loading it all the time. I’m currently using the LocalStorageAdapter for that.
  • Once we have it, we would like to sync changes (polling or using Socket.IO in the background and updating the LocalStorage could do the trick)
  • There’re other models that are more transactional, where we would need to directly go to the Server and get/save them. It would be nice to use something like the RESTAdapter for that.
  • Lastly, there’re some operations that should work off-line and changes should be synced later.

To make it more concrete:

  • We pre-load vendor and “favorite products” into Local Storage. We work offline with those.
  • We need to sync server changes to vendor and product information.
  • If they search the full catalog, that requires them to be online.
  • When offline, we need to allow users to add something to their cart or even submit and order. We would like to queue this action and submit it when they have an Internet Connection.

So a few questions are derived from this:

  • Is there a way to user RESTAdapter in combination with LocalStorage?
  • Is there some Socket.IO support? (Happy to do this part manually)
  • Is there Queueing support? Ideally at the Ember-Data level.

I know we will have to do a lot of this manually and pull together the different lego pieces, but I wanted to ask for some perspective from experience Ember devs.

2 Likes

I’ve actually been thinking about making something like this. I feel like the ideal is to just have a LS fallback mixin, so that any transactions that fail with certain error codes just get stashed in LS, and then synced later. On its face it doesn’t seem that complicated to implement, but I’m sure I’m not considering everything.

1 Like

I am working on a plain JS library to handle synchronization across any number of data sources. These data sources could be REST, local storage, sockets, etc. - they just need to implement standard interfaces which allow them to be connected together. I am currently building a bridge to Ember Data in the adapter layer.

I hope this doesn’t seem like a tease to mention without a link, but I’m very close to open sourcing this library and I think it should handle your use case well.

1 Like

I would love to have a look. I’m interested in this space. That said, IMO, the fallback is too magical. This is something I would rather do it explicitly. For example, some actions should simply not work offline, synching them later would be too complicated or we just don’t have enough data client side to do it (e.g. searching our full catalog). In other cases, we rely on the data always being there and we know it’s relatively static, so we don’t even want to try to go to the server, no need to fallback, the data should always be there and be recent enough up to date (by a webworker). This leaves only few cases where we need to support offline behavior that needs to be synched later, for those I think it’s better to be explict. Also instead of falling back, we could detect the state of the connection and use one or the other adapter… This might be just my use case or personal preference based on other technologies I used in the past, but I guess others would find that fallback approach valuable.

Looking forward to it. I’ve been doing it by hand now just to be able to iterate fast with it. Once I identify the patterns I might extract some of this into a reusable form. At the moment I’ve been mainly using a repository pattern to extract all of the data access logic away. Different repositories get their data in different ways. I think the repositories could become ED adapters, but at the moment the interface is too different (e.g. I’ve a method to get deltas from the server for a set of models at once, then push them to the store).

Maybe some news in 2017? Ember-data offline data persistence, maybe ember-pouch is not useful? Maybe I need something else?