Routing is not working for large data from api

Using

"ember-data": "^3.5.0",
"ember-cli": "~3.5.1",
"ember-cli-app-version": "^3.2.0",
"ember-cli-htmlbars": "^3.0.0",

I have a link on the page that triggers an API when it is clicked if the response is small let say 2-3 objects it routes to that pages successfully but if the data is large 2-3 MB of data it doesn’t route to that page but if I open it on the new tab it opens it.

here is the link code

 {{#link-to "rule-sets.rule-set-versions.index" ruleSet.id class="btn btn-link"}}View
                                    Versions{{/link-to}}

Thanks.

What does your model hook look like on your target page?

I didn’t get you I am just starting to work on ember can you please tell me what is that?

Sure so you said this is the link that, when clicked, doesn’t route to the page for a large payload (but does if it’s a new tab) right? This link is pointing to the rule-sets.rule-set-versions.index route as its “target” and passing ruleSet.id as its dynamic segment. Presumably that route is fetching the data in its model() hook, so I was wondering what the model hook looked like in that route. That may help diagnose the problem… It also may not but it’s the first thing I would check.

I guess this is what you are talking about right

export default Route.extend(ResetScrollMixin, {

  model(params) {
    return Ember.RSVP.hash({
      ruleSetVersions: this.store.query('rule-set-version', {
        rule_set_id: params.rule_set_id,
        is_archived: false
      }),
      ruleSet: this.store.find('rule-set', params.rule_set_id)
    })
  }
});

Hmmm ok this line isn’t quite right. The store.find method is private (it was public a long time ago but hasn’t been used that way since early Ember 2.x at least IIRC) and you should use store.findRecord instead. I kinda doubt that will be breaking anything but it’s possible. Are there any errors in the javascript console when this happens? Or does it just hang and never “finish” the transition? Or something else…?

No there is no error in the console but yes the transition doesn’t take place even if the request is completed in some cases.

And about the query it is working find as the ruleset is getting populated but I can check it out putting findRecord.

Thank you so much for taking out time for this :slightly_smiling_face:

I would definitely use findRecord instead just to make sure you’re using public API and rule out future issues with that even if it’s working right now (I assume this was an older app that was updated?).

Anyway as to the main issue you’re having… if there aren’t any errors it kinda sounds like something is just going really slow with large payloads. Partly that seems obvious, the bigger the payload the longer it takes to send/serialize/render. But in theory this still should work fine even if it’s slow. Roughly how many records would you say are loaded in the 2-3mb payload? There are a number of parts that could be slow here and it’s important to figure out which one:

  • network request - how long does the network request take? probably not super long
  • ember data serialization - transforming the records from raw JSON to DS.Model instances and loading them in the store takes time and lots of code execution, with a great many records this could be very slow, but I would expect it to take thousands of records before it would be very noticeable
  • rendering time - sometimes the rendering time is prohibitive, the best way to measure this is probably the ember inspector extension in Chrome, go to the page and look at the “rendering” tab. It will create a tree of component render times.
  • if this isn’t turning anything up you could look at your Chrome dev tools, specifically the Performance tab (maybe memory too if you’re noticing big memory spikes). I recently was debugging an issue where receiving a large number of records via websocket was super slow (and would sometimes crash) and found the cause digging into a snapshot in the performance inspector. It turned out to be a computed property that was recomputed thousands of times which caused thousands of re-renders, etc.

Mitigation strategies will depend on what’s taking the most time, so if you can figure out where the bulk of the execution time is coming from you can probably optimize it.

The size of the response is almost 4MB.

I have checked the ember inspector and found that even if the API response is successful the data section is not getting populated in the inspector that could be the case.

And also the response for 1 API is 3-4 sec which is not loading data and the other one takes 30-40 sec.

My serializer looks like this

import DS from 'ember-data';


var underscore = Ember.String.underscore;
export default DS.RESTSerializer.extend(DS.EmbeddedRecordsMixin, {
  attrs: {
    rules: {
      embedded: 'always'
    },

    test_cases: {
      serialize: "id",
      deserialize: 'records'
    },
  },
  keyForRelationship: function (rawKey) {
    return underscore(rawKey) + "_id";
  }
})

The response looks like this and I am unable to use the pagination variables as well

{
	"page_number": 1,
	"page_size": 5,
	"rule_set_versions": [{
		"id": 930,
		"last_updated_by": 1,
		"created_by": 1,
		"default_rule_id": 963,
		"is_live": true,
		"is_archived": false,
		"start_time": "2020-12-30T18:30:00Z",
		"rule_set_id": 11,
		"rules": [{
			"id": 963,
			"priority": 10000001,
			"conditions": [],
			"results": [{
				"id": 59,
				"operator": null,
				"value": "00",
				"rule_id": 963,
				"rule_set_metadatum": {
					"created_at": "2020-11-18T10:01:38Z",
					"datatype": "integer",
					"id": 34,
					"name": "edd_result",
					"property_category": "result",
					"updated_at": "2020-11-18T10:01:38Z",
					"validations": {
						"max": 99999,
						"min": 0
					}
				},
				"created_at": "2020-11-18T10:10:41Z",
				"updated_at": "2020-11-18T10:10:41Z"
			}],
			"created_at": "2020-11-18T10:10:32Z",
			"updated_at": "2020-11-18T10:10:32Z"
		}, {
			"id": 971,
			"priority": 1,
			"conditions": [{
				"id": 57,
				"comparator": "equal_to",
				"value": "100",
				"rule_id": 971,
				"rule_set_metadatum": {
					"created_at": "2020-11-18T10:01:05Z",
					"datatype": "text",
					"id": 33,
					"name": "edd_con",
					"property_category": "condition",
					"updated_at": "2020-11-18T10:01:05Z",
					"validations": {}
				},
				"created_at": "2020-12-23T09:05:04Z",
				"updated_at": "2020-12-23T09:05:04Z"
			}],
			"results": [{
				"id": 66,
				"operator": null,
				"value": "100",
				"rule_id": 971,
				"rule_set_metadatum": {
					"created_at": "2020-11-18T10:01:38Z",
					"datatype": "integer",
					"id": 34,
					"name": "edd_result",
					"property_category": "result",
					"updated_at": "2020-11-18T10:01:38Z",
					"validations": {
						"max": 99999,
						"min": 0
					}
				},
				"created_at": "2020-12-23T09:05:10Z",
				"updated_at": "2020-12-23T09:05:10Z"
			}],
			"created_at": "2020-12-23T09:04:54Z",
			"updated_at": "2020-12-23T09:04:54Z"
		}],
		"created_at": "2020-11-18T10:10:32Z",
		"updated_at": "2020-12-23T09:06:04Z"
	}, {
		"id": 932,
		"last_updated_by": 1,
		"created_by": 1,
		"default_rule_id": 969,
		"is_live": true,
		"is_archived": false,
		"start_time": "2020-11-25T18:30:00Z",
		"rule_set_id": 11,
		"rules": [{
			"id": 969,
			"priority": 10000001,
			"conditions": [],
			"results": [{
				"id": 64,
				"operator": null,
				"value": "1000",
				"rule_id": 969,
				"rule_set_metadatum": {
					"created_at": "2020-11-18T10:01:38Z",
					"datatype": "integer",
					"id": 34,
					"name": "edd_result",
					"property_category": "result",
					"updated_at": "2020-11-18T10:01:38Z",
					"validations": {
						"max": 99999,
						"min": 0
					}
				},
				"created_at": "2020-11-25T11:53:58Z",
				"updated_at": "2020-11-25T11:53:58Z"
			}],
			"created_at": "2020-11-25T11:53:49Z",
			"updated_at": "2020-11-25T11:53:49Z"
		}, {
			"id": 970,
			"priority": 1,
			"conditions": [{
				"id": 56,
				"comparator": "equal_to",
				"value": "100",
				"rule_id": 970,
				"rule_set_metadatum": {
					"created_at": "2020-11-18T10:01:05Z",
					"datatype": "text",
					"id": 33,
					"name": "edd_con",
					"property_category": "condition",
					"updated_at": "2020-11-18T10:01:05Z",
					"validations": {}
				},
				"created_at": "2020-11-25T11:54:11Z",
				"updated_at": "2020-11-25T11:54:11Z"
			}],
			"results": [{
				"id": 65,
				"operator": null,
				"value": "100",
				"rule_id": 970,
				"rule_set_metadatum": {
					"created_at": "2020-11-18T10:01:38Z",
					"datatype": "integer",
					"id": 34,
					"name": "edd_result",
					"property_category": "result",
					"updated_at": "2020-11-18T10:01:38Z",
					"validations": {
						"max": 99999,
						"min": 0
					}
				},
				"created_at": "2020-11-25T11:54:16Z",
				"updated_at": "2020-11-25T11:54:16Z"
			}],
			"created_at": "2020-11-25T11:54:03Z",
			"updated_at": "2020-11-25T11:54:03Z"
		}],
		"created_at": "2020-11-25T11:53:49Z",
		"updated_at": "2020-11-25T11:55:12Z"
	}, {
		"id": 934,
		"last_updated_by": 1,
		"created_by": 1,
		"default_rule_id": 974,
		"is_live": false,
		"is_archived": false,
		"start_time": "2020-12-30T18:30:00Z",
		"rule_set_id": 11,
		"rules": [{
			"id": 974,
			"priority": 10000001,
			"conditions": [],
			"results": [{
				"id": 69,
				"operator": null,
				"value": "00",
				"rule_id": 974,
				"rule_set_metadatum": {
					"created_at": "2020-11-18T10:01:38Z",
					"datatype": "integer",
					"id": 34,
					"name": "edd_result",
					"property_category": "result",
					"updated_at": "2020-11-18T10:01:38Z",
					"validations": {
						"max": 99999,
						"min": 0
					}
				},
				"created_at": "2021-03-11T10:55:44Z",
				"updated_at": "2021-03-11T10:55:44Z"
			}],
			"created_at": "2021-03-11T10:55:44Z",
			"updated_at": "2021-03-11T10:55:44Z"
		}, {
			"id": 976,
			"priority": 1,
			"conditions": [{
				"id": 60,
				"comparator": "equal_to",
				"value": "100",
				"rule_id": 976,
				"rule_set_metadatum": {
					"created_at": "2020-11-18T10:01:05Z",
					"datatype": "text",
					"id": 33,
					"name": "edd_con",
					"property_category": "condition",
					"updated_at": "2020-11-18T10:01:05Z",
					"validations": {}
				},
				"created_at": "2021-03-12T16:18:04Z",
				"updated_at": "2021-03-12T16:18:04Z"
			}],
			"results": [{
				"id": 71,
				"operator": null,
				"value": "100",
				"rule_id": 976,
				"rule_set_metadatum": {
					"created_at": "2020-11-18T10:01:38Z",
					"datatype": "integer",
					"id": 34,
					"name": "edd_result",
					"property_category": "result",
					"updated_at": "2020-11-18T10:01:38Z",
					"validations": {
						"max": 99999,
						"min": 0
					}
				},
				"created_at": "2021-03-12T16:18:10Z",
				"updated_at": "2021-03-12T16:18:10Z"
			}],
			"created_at": "2021-03-12T16:17:55Z",
			"updated_at": "2021-03-12T16:17:55Z"
		}],
		"created_at": "2021-03-11T10:55:44Z",
		"updated_at": "2021-03-11T10:55:44Z"
	}],
	"total_entries": 3,
	"total_pages": 1
}

The only problem I am facing here is unable to use the pagination variables

Hi @dknutsen Any suggestions for this serializer issue, please.

Sorry for delay wasn’t around my computer much this weekend. I would try using the extract_meta method on the serializer. You could pull those keys from the payload and return them in an object, which would then be set on “meta” on the record/queryResult.