Best way to handle duplicated rows

Question

I have insurance companies "dictionary" in my database, let's say:

+----+-------------------+----------+
| ID | Name              | Data     |
+----+-------------------+----------+
| 1  | InsuranceCompany1 | SomeData |
+----+-------------------+----------+

But I'm fetching data from another system, and in result I got duplicates of insurance companies, but without my data:

+----+-------------------+----------+
| ID | Name              | Data     |
+----+-------------------+----------+
| 1  | InsuranceCompany1 | SomeData |
+----+-------------------+----------+
| 2  | InsuranceCompany1 |          |
+----+-------------------+----------+

Both records are related in variety of models but they refer to the same data, and what I want is to pair these records without changing queries or data in other tables, so noone knows there are two records, but both refer to one instance which is

 +----+-------------------+----------+
 | 1  | InsuranceCompany1 | SomeData |
 +----+-------------------+----------+

My question is: Is there some proper way to handle situations like this? I've came up with solution which is to add parent_id column, and manually set parent_id in duplicated rows, and then override Eloquent methods like find in a model to return parent if there is parent_id set.

Copying SomeData column is not an option because there can be condition if insurance_company_id == id;

O. Jones · Answer 1 · 2020-05-25T11:10:47.793

You can try creating a view of your dict table something like this:

  CREATE VIEW unique_dict AS
  SELECT MIN(ID) ID,
         Name,
         GROUP_CONCAT(Data) Data
    FROM dict
   GROUP BY Name

That will give you one row per name.

Then, in your queries requiring one row per name, SELECT from the unique_dict view rather than the dict table.

GROUP_CONCAT() yields a list of values from Data, which helps if more than one duplicated row contains a value: you get them all.

Longer term you might be smart to consider these duplicates to be "dirty data", and clean them up as you INSERT new rows. How to do that?

Create a unique index on Name.

CREATE UNIQUE INDEX unique_name ON dict(Name);

Then, when loading new data into dict use Eloquent's updateOrCreate() function. Here's something to read about that. Laravel 5.1 Create or Update on Duplicate

Best way to handle duplicated rows

1 Answers1