Lazily load and render a lot of data


#1

Good evening.

Sorry for my bad English.

I have an API that provides items and subitems:

/api/items
/api/items/1/subitems

Both of this endpoints can support offset and limit for example.

Items and subitems are displayed in tables.

What is the best way to lazy load and display a lot of rows (~200 000 of items and many of subitems).

Do I need to provide custom action for scroll and load data?

I know about List-View. Can I customize it to support tables and connect scroll event or something to load more data?

Thanks a lot.


#2

You have two alternatives as far as I know:

  • use list view. but im not sure if it could be automatically wired up with a table.
  • use group helper which is what i have been doing. you will run issues with loading 200 000 items at once so you will need to implement some infinite scrolling yourself which is quite simple.

#3

List-View is currently not able to display variable row heigths. If this is not an issue for you you can use list-view. The other workaround is to use an infinite-scrolling collection that appends new data to the dom when you reach the end of the list or by having the user trigger an action (by pressing a button for example). There’s also ember-table from adeepar which might be exactly what you are looking for.

200 000 items is really a lot of data. Loading thousands of records is not a problem on the controller layer but when rendering the records to the dom. When working with hundreds of thousands of records this may be an issue on the controller layer as well but I am not sure if this is really the case. I have a use case where we might display thousands of records but this will be less than 10000 and the controller layer (sorting etc.) is not an issue.

The nice thing about an infinite collection is the possibility to display variable heights in a list and it is pretty straight forward to do yourself. Also you have the possibility to use ember-cloaking to make sure you are not running out of memory when displaying large lists.

I tried to customize list-view for variable row heights but it is kind of complicated and I am still working on that. The easiest way to render long list is taking the infinite scrolling approach an rendering small chunks of dom-elements to the dom while the user is scrolling.


#4

Thanks everyone for the help.