What is going on with the new JSON Format and the Serializers?


#1

TL;DR: super call in normalizeArrayResponse returns JSON Api Format, while being passed the old JSON format to handle.

Long explanation start

I have a Rest server which returns a findAll request like this (say for example querying Accounts):

{
    [
        {
            "id": "5",
            "name": "name 1"
        },
        {
            "id": "8",
            "name": "name 1"
        }
    ]
}

With the new Serializer and JSON Api format, you would have the normalizeArrayResponse return a JSON Api format of the above request like this, in order to adhere to the format.

{
    "data": [
        {
            "id": "5",
            "type": "account",
            "attributes": {
              "name": "name 1"
            },
            "relationships": {}
        },
        {
            "id": "8",
            "type": "account",
            "attributes": {
              "name": "name 1"
            },
            "relationships": {}
        }
    ],
    "included": []
}

So for that, I would write my serializer something like this (mind the flag isNewSerializerAPI: true, is being set).

import DS from 'ember-data';

export default DS.RESTSerializer.extend({
  isNewSerializerAPI: true,

  normalizeArrayResponse: function(store, primaryModelClass, payload, id, requestType) {
    var normalizedRecords = [];

    payload.map(function(record){
      var id = record.id;
      delete record.id;

      var normalizedRecord = {
        'type': primaryModelClass.modelName,
        'id': id,
        'attributes': record
      };

      normalizedRecords.push(normalizedRecord);
    });
  
    normalizedRecords = {data: normalizedRecords};
    return normalizedRecords;
   }
});

This mostly works, however the only problem is, because I return just the JSON document (not calling this._super), there are no Transforms being applied, (I have other attributes next to name and Id on account, where some are custom, and need a transform).

I thought that simply changing return normalizedRecords; with return this._super(store, primaryModelClass, normalizedRecords, id, requestType); would do the trick, but here things get weird.

I was getting a bunch of errors, for example: where the hash data was being singularized, and there was no DS.Model ‘Datum’ defined.

After looking through the source code, and trying to figure out how to either apply transforms to my JSON Api document, or have this._super handle a JSON Api format properly, I managed to figure out that the payload variable has to be in the old format, for it to work.

So when I normalized my JSON document to this:

{
    "account": [
        {
            "id": "5",
            "type": "account",
            "name": "name 1"
        },
        {
            "id": "8",
            "type": "account",
            "name": "name 1"
        }
    ],
}

and using that as the payload attribute in my this._super call in the normalizeArrayResponse hook, everything worked, and my transforms were being applied.

And after debugging, I found out that for some reason, the this._super returns my previous JSON document (in the old format), returns a valid JSON Api document (in the new format)

So the Serializer below normalizes my initial REST Response to the previous JSON-format (see format right above), and then the this._super normalizes THAT to a valid JSON Api Document

import DS from 'ember-data';

export default DS.RESTSerializer.extend({
  isNewSerializerAPI: true,

  normalizeArrayResponse: function(store, primaryModelClass, payload, id, requestType) {
    var normalizedRecords = [];

    payload.map(function(record){
      record.type = primaryModelClass.modelName;
      normalizedRecords.push(record);
    });

    var obj = {};
    obj[primaryModelClass.modelName] = normalizedRecords;


    return this._super(store, primaryModelClass, obj, id, requestType);
   }
});

The Serializer above, returns the json below, because the super call, changes the old format to the new:

{
    "data": [
        {
            "id": "5",
            "type": "account",
            "attributes": {
              "name": "name 1"
            },
            "relationships": {}
        },
        {
            "id": "8",
            "type": "account",
            "attributes": {
              "name": "name 1"
            },
            "relationships": {}
        }
    ],
    "included": []
}

I guess there must be a question now:

  1. How do I apply transforms without calling super?
  2. Am I supposed to call super?
  3. Why doesnt the super class work with the new format (even though the isNewSerializerAPI flag is correctly set?
  4. Am I doing something horribly wrong?

Kind Regards, pjcarly


#2

I believe that the isSerializerAPI flag is just for the different hooks available for you to use on a serializer. the RESTSerializer still expects, however, data in that old format.

I think what you’re trying to do would be to change the 3rd line to

export default DS.JSONAPISerializer.extend({...});

Hope that helps!


#3

I worked together with pjcarly quite a bit on this the past days - even though our problem might not be exactly the same, I am still struggling with the same kind of issues…

Attached is a gist which we both have been discussing back and forth - what it’s trying to do is convert OLD API DATA for the DS.RESTSerializer, to prepare for Ember Data 2.0.

What beats me, and what is most annoying is the fact that I thought Ember would do this for me, but I never made that work properly.

Is the below solution even the correct approach? Is there any way we can help improve the docs around this?

Attached gist, with a serializer example


#4

If you’re using the built-in serializers (RESTSerializer, JSONSerializer, ActiveModelSerializer or JSONAPISerializer) you should not have to manually create the JSON API Document.

The isNewSerializerAPI: true flag does two things:

  1. It tells the store to expect a normalized JSON API Document back from your serializers
  2. It calls the normalizeResponse() flow of your serializer instead of the old extract() flow

All the built-in serializers has both of these flows and can return the old internal Ember Data format as well as a JSON API Document.

With isNewSerializerAPI: true set in your serializer you can chose to do modifications to your payload in the original (as in REST, JSON, ActiveModel) format before calling this.super() or directly in the JSON API Document using the result from this._super().

As long as you make sure that your payload looks like REST, JSON, ActiveModel format before you call this._super() the built-in serializers will take care of converting it to a JSON API Document for you.

  1. How do I apply transforms without calling super?
  2. Am I supposed to call super?

I would strongly recommend you to call this._super() to get things like transforms, type/attribute/relationship normalizations and embedded records for free.

  1. Why doesnt the super class work with the new format (even though the isNewSerializerAPI flag is correctly set?

When calling this._super() on the built-in serializers they always expect the same format as input, regardless of the isNewSerializerAPI flag. You feed it with a REST, JSON or ActiveModel payload and it will transform it to a JSON API Document if the isNewSerializerFlag is set to true.

  1. Am I doing something horribly wrong?

You shouldn’t have to create the JSON API Document manually, but I can totally understand the confusion.


#5

Okay, that definitely clears up issues I had, but what I understood was that starting from 2.0 the old JSON format would be deprecated, but if I read the comments here, depending on the Serializer I use, I would need to feed it the expected JSON format, that seems logical.

But then follows the obvious question, what Serializer do I have to use?

I thought that since I was working with a REST server, the REST Serializer would seem like the obvious choice.

What I can understand from the source code is the following:

  1. RestSerializer extends from JSONSerializer
  2. JsonAPISerializer also extends from JSONSerializer

So here are my follow up questions:

  1. What is the difference between RestSerializer and JSONSerializer?
  2. Would using the the JSONApiSerializer, in combination with the RestAdapter work? (taken I normalize my json to the JSONAPI format in the normalize hooks)
  3. Is there going to be a point where the old JSONSerializer is going to be deprecated (and removed) in favor of JSONApiSerializer
  4. If the previous question is true, what is going to happen with RestSerializer?

#6

What is the difference between RestSerializer and JSONSerializer?

The RESTSerializer normalizes/serializes from/to a format usually suitable for REST backends. Like:

{ users: [{ id: 1, firstName: 'Christoffer', lastName: 'Persson' }] }

The JSONSerializer is a “simpler” serializer that only handles one type per response. The equivalent of the payload above would be:

[{ id: 1, firstName: 'Christoffer', lastName: 'Persson' }]

Would using the the JSONApiSerializer, in combination with the RestAdapter work? (taken I normalize my json to the JSONAPI format in the normalize hooks)

That would probably work (EDIT: I wouldn’t recommend it) but you should chose adapter/serializer based on your server’s API. If you have a REST-like API you should use the RESTAdapter/RESTSerializer. Just pass your REST payload to the RESTSerializer and it will normalize to JSON API for you.

Please note that it’s only the internal representation of data that has changed. You are in no way required to convert your server API to JSON API or manually munge your payloads to JSON API if you’re using any of the built-in serializers.

Is there going to be a point where the old JSONSerializer is going to be deprecated (and removed) in favor of JSONApiSerializer

I don’t think so. In a worst case scenario they would be moved to addons.


#7

Hi @pjcarly. Allow me to try to answer your questions.

  1. What is the difference between RestSerializer and JSONSerializer?

The difference between the RESTSerializer and the JSONSerializer is the format of the JSON payload they expect the server to understand and consumer. The RESTSerializer always namespaces serialized payloads by their type in the payload, and can support side-loaded records. Its format looks something like this:

{
  "post": {
    "id": 1,
    "title": "Rails is omakase"
  }
}

The JSONSerializer does not namespace its payload so it can only contain the primary record type in its response. It looks something like this.

{
  "id": 1,
  "title": "Rails is omakase"
}

When reading a response from the server both these serializers are responsible for transforming the server response payload into a JSON-API document that Ember Data’s store.push API can understand. When Ember Data talks about removing the old JSON format in 2.0 its referring to the old “normalized json” format that serializers use to “push” into the store. This format was the very ad hoc, poorly documented and had several gotchas when combining advanced features of Ember Data. The hope is that by switching store.push to a better-specified, better-documented format it will be easier for users to create new serializers for talking with different backends.

The flow of serializing records for sending them to the server does not change. The serializers are still given a snapshot of a record and are expected to use that snapshot to create a JavaScript Object that represents a serialized version of the record that the server can understand.

  1. Would using the the JSONApiSerializer, in combination with the RestAdapter work? (taken I normalize my json to the JSONAPI format in the normalize hooks)

Yes, Combining the JSONAPISerializer and RESTAdapter should work fine.

  1. Is there going to be a point where the old JSONSerializer is going to be deprecated (and removed) in favor of JSONApiSerializer

I don’t see the JSONSerializer being removed. It supports different backends then the JSONAPISerializer so they will both continue to be supported by Ember Data. It also is a much simpler format (and supports less features) which makes it a nice starting point for extending to build other serializers.

  1. If the previous question is true, what is going to happen with RestSerializer?

The RESTSerializer will continue to be supported for the life of Ember Data 2.0. Although the defaults, and documentation will encourage developers starting new projects to build backends that support the JSONAPISerializer.


#8

Cheers guys, this made everything clear


#9

Hey guys!

I am sorry for hijacking this thread a bit - I just want to clear this up for my own sake - spent a little over 60 hours on this and it seems I just misunderstood it all… :smiley:

  • I clearly do not need the gist linked above? Gist link again
  • Is there any way any of you guys have a snippet of code that could explain how to accomplish this: Old API format from API (including sideloading) -> Ember data/Ember 1.1.13 (and coming 2.0)

I wrote a blogpost as well around this which I probably should archive? :smile: It’s linked from the gist.

Edit: I understand the above expenation from @bmac - but I simply could not accomplish an implementation. I can’t understand what I am doing wrong. What I need to have is serializers prelared for Ember 2.0 (Ember data 1.11.13) working with old data. :frowning:


#10

@hussfelt Do you mind sharing a gist of what your server’s response payload looks like?


#11

Thanks for all the help! This gist made it clear:


#12

@bmac It was the old format, I just misunderstood how the new hooks where implemented. :frowning:

Just replacing the old with the new and calling super fixed most things!