Streaming data questions

[Montlebalm]

Why hello there. I’m attempting to create a page that streams stock quotes in (simulated) real-time. It works, but I’m certain it’s not using the best abstractions for Ember.

The full repo is here, but I’ll post some examples below.

Architecture

Streamer

This class is responsible for providing the real-time updates via websockets or long-polling. It’s currently faked, but it should be abstracted to allow any type of connection.

I don’t feel like this should be an Ember.Object. Maybe an Adapter?

var Streamer = Ember.Object.extend({
  onConnect: function(data) {},
  onUpdate: function(updates, date) {}, // Client callback
  schedule: function() {
    return Ember.run.later(this, function() {
      // Call `onUpdate` with new data
      this.set('timer', this.schedule());
    }, 1000);
  },
  start: function() {
    this.set('timer', this.schedule());
  },
  // ...
});

Route

The route queries the store for the initial set of stock symbols that will be passed to the Streamer. It’s also responsible for instantiating the Streamer and setting the real-time data on the controller.

var StocksRoute = Ember.Route.extend({
  model: function() {
    return {
      symbols: this.store.find('symbol')
    };
  },
  setupController: function(controller, model) {
    controller.set('model', {
      lastUpdatedDate: null
    });

    model.symbols.then(function(obj) {
      var streamer = Streamer.create({
        onUpdate: function(updates, date) {
          controller.set('updates', updates);
          controller.set('model.lastUpdatedDate', date);
        },
        symbols: model.symbols.toArray()
      });
      streamer.start();
    });
  },
  // ...
});

Controller

Finally, the controller receives the data from the route and translates it into something that can be rendered in the template. It does this by watching the updates property that is updated by the route. None of the streamed data will be modified by the controller, so it never needs to update any models.

It feels like regenerating the entire stocksDisplay property each update is inefficient. I could probably refactor this to be cleaner.

var StocksController = Ember.ObjectController.extend({
  stocks: [], // Set by route
  stocksDisplay: function() {
    var updates = this.get('updates');

    return _.map(this.get('stocks').toArray(), function(stock) {
      var update = updates[stock.symbol] || {};
      var display = {
        name: stock.name,
        symbol: stock.symbol,
        values: {
          ask: {
            value: stock.ask,
            updated: false
          },
          // ...
        }
      };

      // Update the values
      var props = _.pick(update, 'ask', 'bid', 'price', 'volume');
      _.each(props, function(value, key) {
        stock[key] = display.values[key].value = value;
        display.values[key].updated = true;
      });

      return display;
    });
  }.property('updates'),
  updates: {}, // Set by route
  // ...
});

Questions, summary, etc

1. What kind of object should the Streamer be? It feels wrong in its current form, but I don’t know how else it should be structured.
2. Is it appropriate for the route to control the streaming data?
3. Should any of this controller logic be placed into a Ember.View?
4. Does my use of properties vs model look correct on the controller?
5. Are there any performance concerns with my approach? The Ember Inspector says updating 10 stocks takes ~4ms.

I know that’s a lot to digest. Please ask any clarifying questions if I’ve left anything too ambiguous.

Thanks.

[mitchlloyd]

I think your Streamer object is a perfect use case for an Ember.Service. Check out:

Services: A Rumination on Introducing a New Role into the Ember Programming Model

Many Ember apps need to pull in additional functionality beyond what’s provided out of the box. Often times, these additional dependencies don’t always fit neatly into the existing roles provided in the programming model. Some examples of things I’ve needed to implement in my own apps: WebSocket initialization Geolocation Global config (for things like feature flags, etc.) Most people end up putting this extra functionality in a controller, which is great, and works in many cases. For exampl…

and current Ember code. Service is behind a feature flag in Canary. I would also include Ember.Evented in that service to trigger events.

I would probably use Ember.inject.service inside of the Route to give it access to the service. Then in your Route you can setup callbacks using on inside of a setup function:

  setupStreamerCallbacks: function() {
    this.get('streamer').on('updateWasReceived', this, 'stockUpdateWasReceived');
  }.on('init')

Then it is pretty temping to look for a way to use Ember Data in the Route to update stock models that you give to the controller in the model hook. I’m a little lost on how the controller takes symbols and stocks, but it looks like you know what you’re doing.

In the callback in your Route it would be nice to be able to do this:

  stockUpdateWasReceived: function(stockData) {
    // somehow get your data into a good form or make sure your service returns something good
    this.store.pushPayload('stock', stockData);
  }.

And then just watch your page update through the beauty of one-way data binding

Without Ember Data, I think it is fine to keep a cache of stocks in your controller and then call a method on the controller to give it an update.

  stockUpdateWasReceived: function(stockData) {
    this.controller.handleStockUpdate(stockData);
  }.

Regarding performance: I wouldn’t worry about it too soon. Also remember that if you are looking at this in development, it should be a lot faster in production with a production Ember build.

Regarding Ember.View: The party line on Views is “Don’t use Ember.View for application code. Use a component instead”. I don’t see anything in this controller that would be better in a component.

[Montlebalm]

Thanks. There are some great suggestions in there.

I played around with upgrading my ember-cli project to canary, but I wound up reverting after running into problems. I’ll give it another go sometime soon and try out the official services.

For now, I added Ember.Evented to the streamer and refactored the instantiation parameters into proper events. It’s cleaner and gives greater control.

I wasn’t able to figure out how to push to the store such that the controller automatically re-rendered the updates. I tried using store.pushMany('stock', stocks), but nothing changed. I’ve resorted to the old way of controller.set('model.stocks', stocks) during the stream updates.

Btw, is it normal that I need to convert my models to json and add the id before pushing to the store? It seems like I ought to be able to pass in the models themselves.

[mitchlloyd]

Nice! I’m glad there was something helpful in there.

I’m poking around in your repo a little more and I see that your streamer is simulating a payload with all the stocks. I think the way you’re doing this without using the store makes perfect sense here since the entire set of stocks could be changing each push.

One minor note in your StocksRoute: Going forward ObjectController is going to be deprecated and in this case I think your code would be a little clearer if it just used a plain Controller for the StocksController. Then in your route, you could keep your model hook the same, but just set lastUpdatedDate and 'stocks` as properties directly on the controller.

I was a little surprised to see that streaming-cell was a helper rather than a component. I think this would be a better fit for a component since it is creating HTML for these streaming cells with specific classes (see classNameBindings). This would also avoid the safeString, escapeExpression dance.

[Montlebalm]

You might have been right the first time regarding pushing updates to the store. The stocks that are streaming won’t change update-to-update, just their values.

I converted the helper to a component per your suggestion. I think it’s cleaner, but it definitely slowed rendering (4ms to 30ms). Not a big deal, but it could get problematic if you add more stocks and data points. It should naturally get faster as Ember gets better at recognizing when components don’t need to be re-rendered.

The classNameBindings is an interesting way to solve the conditional class problem. It struck me as odd at first, but I can see its elegance.