I know there is a ton of same questions about finding and removing duplicate values in mySQL
but my question is a bit different:
I have a table with columns as ID
, Timestamp
and price
. A script scrapes data from another webpage and saves it in the database every 10 seconds. Sometimes data ends up like this:
| id | timestamp | price |
|----|-----------|-------|
| 1 | 12:13 | 100 |
| 2 | 12:14 | 120 |
| 3 | 12:15 | 100 |
| 4 | 12:16 | 100 |
| 5 | 12:17 | 110 |
As you see there are 3 duplicated values and removing the price
with ID = 4
will shrink the table without damaging data integrity. I need to remove continuous duplicated records except the first one
(which has the lowest ID
or Timestamp
).
Is there a sufficient way to do it? (there is about a million records)
I edited my scraping script so it checks for duplicated price
before adding it but I need to shrink and maintain my old data.