Ember Data, memory, and unloading records


#1

So my colleague was looking into this last week and hit the slack channel and github but we didn’t really get anywhere so I’m going to try again. Sorry for the length but I want to be detailed.

The basic issue

Long story short, we have a financial application which uses tons of Ember Data records and that leads to rapid memory buildup in certain views. We are trying to clear unused records out of the store to reclaim some of this memory but while unloading records seems to remove them from the store it doesn’t free the memory (seemingly because of relationships with data that isn’t unloaded) and this is turning into a huge problem for us. I don’t want to be melodramatic but we have thousands of customers on a wide variety of hardware and this cripples even a beefy machine pretty quickly, rendering the application more or less unusable without a refresh or browser restart.

Many users may have the app open for hours at a time and the memory buildup from regular store usage gets out of hand. Obviously we can (and will) make some improvements around how much data gets loaded and put in the store to begin with but at the end of the day we need to be able to clear out old records. Ember Data is holding memory hostage and we need to free it.

Our data model

There are probably four models relevant to this issue. I’ll call them:

  • group
  • subgroup (belongsTo group, less than 10 subgroups in any group)
  • item
    • belongsTo subgroup, up to thousands of items per subgroup)
    • 8x belongsTo pricedata, so 8 pricedata records per item
    • belongsTo item (relationship to self, basically parent/child items)
  • pricedata (belongsTo item, so this is a 2-way relationship)

In some of the problematic views we’ll have ~600 “item” records, then for each of those we’ll of course be loading 8 “pricedata” records (and updating them every 500ms or so). So if a user loads a couple of these problem views in succession (very common use case) we’re easily looking at 2-5k “item” records and 20k+ “pricedata” items. This, along with the other memory allocations for the DOM and Ember app itself, all told, puts the memory usage of this application (from the Chrome task manager or OSX activity monitory) at >1gb memory. At this point the application is pretty slow and/or unusable.

What we want to do

What we’d like to do is just unload the “item” and “pricedata” records associated with these views after we leave them in order to reclaim the memory, avoiding the long-term buildup. If we can help it we’d like to keep all “group” and “subgroup” records around.

What we’ve tried

So obviously we tried just unloadAll on “item” and “pricedata” records when we leave one of the routes, but that wasn’t working. Last week when we were first investigating this my colleague (@trumb1mj) made a super stripped down version of our app that loaded a ton of this data and just had a button that deleted it. At first he tried just unloadAll for both “item” and “pricedata”. What it looked like was happening was that while it would actually unload the records it wouldn’t free them until he deleted all of the connected models. He then made some inquiries on github and slack but the general gist seemed to be “maybe that’s how it should work, maybe not, we don’t know yet”.

Next, in our full app, I again tried unloading just “item” and “pricedata” records when leaving one of these problem views. That clears the records but seems to have no effect on memory, as expected (after the findings described above). We also tried looping over the records and doing an unloadRecord followed by a destroy().

Then I tried this.store.unloadAll(); to clear the entire store. This seems to work, at least partly, but it still takes up to a couple minutes to actually GC the memory, and it wipes out everything, including data that we need in other parts of the app. We may be able to work around the latter issue but this approach certainly isn’t ideal and I can’t even tell if it’s actually reliably dereferencing the memory.

I also tried overloading updateRecord in the adapters (we don’t save data to the backend in this way) to prevent .save() from sending a request to the API, and then going through all of the ‘item’ records and unsetting the relationship with ‘subgroup’, saving the records, then after all that has finished processing, deleting the records individually. This also seems to be having some positive effect but it’s difficult to tell if it’s reliable, and it also requires more processing time than just using unloadAll.

The question

So, is there any way to delete “item” and “pricedata” records (and free the memory) without unloading the entire store, preferably without deleting the “group” and “subgroup” records? Could we structure our data differently? Any weird ED hacks? We’ve invested pretty heavily in Ember Data at this point and I really don’t want to try and back away from it but this is a really big problem for us.

I find it difficult to believe that we’re the first ones to have this issue and that this use case can’t be covered by Ember Data. Even if the default behavior is that relationships maintain references to unloaded records I think there should be a way to opt-out. I understand there is a lot of ED work underway and maybe this will help us out but is there anything we can do in the meantime?

TL;DR is there any way, even a dirty hack, to delete records (and free the memory) of Ember Data records that have belongsTo relationships? Preferably without unloading the entire store and all related records.


#2

update to ember-data canary


#3

Tried that a couple days ago (forgot to mention, sorry) after the big PR was merged over the weekend. Ran into a lot of issues with null records and such. Worked around some of them without much issue but some of them seemed like they were internal ED issues. I’ll try again today and see if I can get anywhere.

Also, while we would consider using a canary/beta in prod if it worked well, it would not be ideal. Also open to workarounds in the meantime.


#4

Generally the beta and canary builds are very stable, if you hit null records post-upgrade I would really like a bug report ASAP unless you can track it down to your own monkey-patching.

I was asking because the PR over the weekend to free up internal models should have helped you out.


#5

I gotcha, thanks for the responsiveness. So what I did today was make a brand new Ember app with essentially a stripped down version of our models that I described above. I used mirage to create a bunch of records and then i have a button that loads all of the records, and then buttons for unloading all records of each type, as well as a button for unloading the entire store. I did some testing with both Ember Data 2.10 and with canary. Here are the results:

Ember Data 2.10

So first I load the app and take an initial heap snapshot. No “InternalModel” allocations as expected.

Next I hit the “Load Records” button which does a findAll on all four model types. My mirage scenario creates 1 group, 2 subgroups, 200 items per subgroup, 8 pricedata records per item for a total of 3603 records. Taking a heap snapshot shows the expected number of InternalModel allocations.

Now I try to unload just the Pricedata records and take another heap snapshot. They are unloaded from the store but no appreciable difference in the heap snapshot.

Next I try unloading the Item records. Pretty much the same results.

Last I delete the group and subgroup records and this does the trick. Memory is freed.

So basically this behavior was expected based on all the other testing we did which I described above. It lines up with what we witnessed in our full app.

Ember Data Canary

Next I installed ED canary and tried the same tests. First step load the app and take initial heap snapshot:

Then load the data. Still same as before, so far so good:

Then if I try to clear Pricedata records, nothing seems to happen. I can step through the code and see that it is being executed but the app doesn’t seem to know that the records have been unloaded (via peekAll) and the Ember Inspector still shows them too. Similarly, the heap snapshot remains about the same:

Then if I try deleting the Item records too, I still get no acknowledgement that the app has any different view of the data, but now there are no InternalModel allocations in the heap snapshot, not even the 3 that I would expect to be retained from Group and Subgroup. The heap size also doesn’t seem to be changed much:

So I’m not really sure what’s going on with Canary but unloadAll doesn’t seem to be working quite as expected. Any ideas? If you want to look at the test app it’s at https://github.com/dknutsen/ed-release-memory.


#6

Thanks! I need to discuss this with a few folks but I suspect there’s a path forward for you.


#7

Awesome! That’s the kind of optimism I like to hear :grin:

Thanks a lot for the help!


#8

Any updates on this? I know you’ve got a lot on your plate, just wanted to follow up and see if there’d been any progress.