0

I have a schedular which runs every hour to fetch data from some rest end point and store it into the my application's database table.

But after each call the data is just getting duplicates in the database.

So, I want to know that is there any way to find delta (difference) in the REST api data and in the Database table data ?

So that I can check for those differences and store those data only into the database table.

Devendra Anchal
  • 289
  • 1
  • 3
  • 4

1 Answers1

0

Creating a diff from the last JSON (fetched from the database) and the current (fetched from the REST api) should help. Either you can skip the data completely if there is no difference, store the new version or just the diff. Creating a diff is described in this answer to How to compare JSON documents and return the differences with Jackson or Gson? .

If you just store diffs, it would be harder to calculate the next diff as you would have to get all diffs from the database, calculate the resulting JSON and then create a diff to the current one.

If the REST api support the HTTP header If-Modified-Since would help a lot.

Arne Burmeister
  • 20,046
  • 8
  • 53
  • 94