2

I have Express running in a Node.js server, which serves as a backen for my React frontend application.

The frontend application fetches data from the backend (which is stored in Mongo) through a REST call, and display this data in a table. The amount of data is growing by the day, so I though I should look into reducing the abount of data transferred to the frontend application, so avoid unnecessary strain on the backend.

I'm not sure if this is the right way to approach this, but I've been thinking I would look into having the backen fetch a limited amount of entries, so that only these data will be displayed in the frontend table.

The problem arises with searching - when the user wants to search the data in the table, I'll need to be able to search through all entries, not just the data loaded into the table.

I guess one option would be to have the search function actually query the REST API, instead of searching the table itself.

If I'm on the right track, I guess I could implement REST API pagination, somewhere along the example found in https://refactoringfactory.wordpress.com/2012/09/08/pagination-in-node-js-and-express/. Other suggestions on how to implement pagination are welcome.

I'd very much like some input on the approach I described, and suggestions for smarter ways implement this.

EDIT: I changed the title somewhat to include "Infinite scroll pagination". This is what I'm looking to implement. At the moment I have a click on pages pagination setup, but would like to replace this for the infinite scroll pagination.

kenneho
  • 401
  • 1
  • 7
  • 24

2 Answers2

3

I've been thinking I would look into having the backen fetch a limited amount of entries, so that only these data will be displayed in the frontend table.

This is common practice in my experience. The term for it is "pagination." Have a look at this SO question regarding best practices for pagination in REST API's: API pagination best practices.

The problem arises with searching - when the user wants to search the data in the table, I'll need to be able to search through all entries, not just the data loaded into the table.

I guess one option would be to have the search function actually query the REST API, instead of searching the table itself.

Again, you got it. Doing small filters/searches on the client is fine for a limited number of entries, but if you need to only retrieve items matching search criteria in the first place, then adding that functionality to your REST API is the right choice.

Community
  • 1
  • 1
TW80000
  • 1,507
  • 1
  • 11
  • 18
  • Thanks for the reply! I'll read the post you linked to, and hopefully the questions I have now will then be answered. To other readers: Just after TW80000 wrote his reply, I edited my original post title and body to include "pagination". – kenneho Aug 24 '17 at 17:46
  • It looks like the accepted answer (https://stackoverflow.com/a/13905523/6650955) is the way to go. My understanding is that the REST API return a data structure containing both data and the cursor (previous/next), and I need to tweak my clients accordingly. Right? I came across https://www.npmjs.com/package/react-infinite-scroller, which I'm hoping I can make use of. – kenneho Aug 24 '17 at 18:26
  • Yes, that's what I would do. Very nice, hopefully it helps. – TW80000 Aug 24 '17 at 18:31
3

Right, you should do

  • pagination: you might implement it by exposing 2 arguments in the rest endpoint for the listing

    • ?p=<number>: page number, defaults to 1
    • ?l=<number>: number of items per page / page length, defaults to a number maybe from 10 to 100

  • search: implement it by exposing 1 argument in the rest endpoint for the listing

    • /?q=<string>: you can define to be what you want, maybe a string that matches with one or multiple fields of the data

If you want to minimize the network traffic, you might also add one more parameter to explicitly select the fields you want to be returned, like this

  • /?f=<string>: string could be something like id,name,age, and so the api should return only those three fields per record.

All this parameters should be accepted by a list endpoint in your RESTful API

Example:

http://example.com/api/cars/?p=2&l=15&q=toyota&f=id,brand,model,color

jperelli
  • 6,988
  • 5
  • 50
  • 85
  • Thanks, that's some great info. I appreciate you providing examples such as these, I'm learning a lot about REST endpoints. – kenneho Aug 26 '17 at 17:33