SQL do not insert duplicates

Question

I have been looking for a solution to best implement a do not insert if the line already exists. I've read a lot of answers which al differ in some way and it's a little above my level to make any sense.

For example the following post is the top result: SQL Server Insert if not exist But I cannot understand why the variables are used and how I define table1 and table2. Would somebody be able to explain this a litle further so that I can apply it to my situation?

In my example I have about 5 columns (Date, Number1, Number2, Text1, Text2) that I'd like to insert from table1 to table2, but only if the do not exist. Update is not necessary. (Therefore merge is out of the question)

We need to know your table structures (of both tables) to help you. Sample data and your definition of "duplicates" would also be needed. — Siyual, Dec 16 '16 at 14:30

score 2 · Answer 1 · answered Dec 16 '16 at 14:44

An alternative to using NOT EXISTS, would be to use the EXCEPT keyword to do a "difference" between the data in the two result sets, selected from both Table1 and Table2:

INSERT INTO table2 (Date, Number1, Number2, Text1, Text2) values 
SELECT A.Date, A.Number1, A.Number2, A.Text1, A.Text2 
FROM Table1 A
EXCEPT
SELECT B.Date, B.Number1, B.Number2, B.Text1, B.Text2 
FROM Table2 B;

score 1 · Answer 2 · edited Jun 04 '17 at 21:00

A not exists should work using a correlated subquery.

The sub select identifies all records in Table1 which do not exist in Table2 by joining on all the fields we would be inserting. This ensures we only insert records which do not already exist in table2.

INSERT INTO table2 (Date, Number1, Number2, Text1, Text2) 
VALUES 
     (SELECT A.Date, A.Number1, A.Number2, A.Text1, A.Text2 
      FROM Table1 A
      WHERE NOT EXISTS (SELECT 1 
                        FROM Table2 B
                        WHERE A.Date = B.Date
                          AND A.Number1 = B.Number1
                          AND A.Number2 = B.Number2
                          AND A.Text1 = B.Text1
                          AND A.Text2 = B.Text2))

Since we don't care what value is selected as it's throwaway from the subquery, I simply select 1 from the not exists correlated subquery.

Now if there are other columns we need to consider, then we would have to add those to our query, or if there's a primary key would could consider instead of all the columns, then we would only need to join on it.

SQL do not insert duplicates

2 Answers2

Linked