Equivalent of MERGE in HIVE query

Question

I have the following SQL query :

MERGE INTO member_staging x
USING (SELECT member_id, first_name, last_name, rank FROM members) y
ON (x.member_id  = y.member_id)
WHEN MATCHED THEN
    UPDATE SET x.first_name = y.first_name, 
                        x.last_name = y.last_name, 
                        x.rank = y.rank
    WHERE x.first_name <> y.first_name OR 
           x.last_name <> y.last_name OR 
           x.rank <> y.rank 
WHEN NOT MATCHED THEN
    INSERT(x.member_id, x.first_name, x.last_name, x.rank)  
    VALUES(y.member_id, y.first_name, y.last_name, y.rank);

I want to implement it in a Hive query , is there any equivalent for MERGE JOIN in HIVE ?

If not in ACID mode, see this answer: https://stackoverflow.com/a/37744071/2700344 — leftjoin, Sep 16 '21 at 10:27
@MartinJames - Come on, first port of call should always be the manual :( https://cwiki.apache.org/confluence/display/Hive/Hive+Transactions — MatBailie, Sep 16 '21 at 11:30

score 5 · Accepted Answer · answered Sep 16 '21 at 10:18

Manual Entry:

https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DML#LanguageManualDML-Merge

Example Syntax:

MERGE INTO <target table> AS T USING <source expression/table> AS S
ON <boolean expression1>
WHEN MATCHED [AND <boolean expression2>] THEN UPDATE SET <set clause list>
WHEN MATCHED [AND <boolean expression3>] THEN DELETE
WHEN NOT MATCHED [AND <boolean expression4>] THEN INSERT VALUES<value list>

Your specific case:

MERGE INTO member_staging AS x
USING (SELECT member_id, first_name, last_name, rank FROM members) y
ON (x.member_id  = y.member_id)
WHEN MATCHED AND (
    x.first_name <> y.first_name OR 
    x.last_name <> y.last_name OR 
    x.rank <> y.rank
)
THEN
    UPDATE SET x.first_name = y.first_name, 
               x.last_name  = y.last_name, 
               x.rank       = y.rank
WHEN NOT MATCHED THEN
    INSERT VALUES (y.member_id, y.first_name, y.last_name, y.rank);

score 0 · Answer 2 · answered Sep 16 '21 at 10:31

The MERGE statement is based on ANSI-standard SQL, so the query is pretty the same. Just switch the WHERE condition in an AND clause like below :

MERGE INTO member_staging AS x
using (SELECT member_id,
              first_name,
              last_name,
              rank
       FROM   members) y
ON ( x.member_id = y.member_id )
WHEN matched AND ( x.first_name <> y.first_name OR x.last_name <> y.last_name OR
x.rank <> y.rank ) THEN
  UPDATE SET x.first_name = y.first_name,
             x.last_name = y.last_name,
             x.rank = y.rank
WHEN NOT matched THEN
  INSERT
  VALUES(y.member_id,
         y.first_name,
         y.last_name,
         y.rank);

Equivalent of MERGE in HIVE query

2 Answers2