plsql procedure to compare two tables where structure of table is not known

Question

So the problem goes like this:

I have a table with two columns namely source query and target query. each row contains a sql query for a informatica mapping from source side and target side and we needed to build a reconciliation procedure which take those two values for each row and generate there output and store it in temp table say temp1 and temp2 and then compare the result of these two temp tables. I Did this by creating two tables and then bulk fetching through two cursors and comparison was done using minus set operator for both the tables and then we got the different rows from there.

Now here comes the tricky part , what we have to do is check those rows which have different value and output the name of the column where there is a change and also output the source side values(temp1) and target side values(temp2).

If I had known the structure of the tables before then yes hard-coding was a way to get there but since the tables temp1 and temp2 are getting dynamically created therefore I'm not able to get my head around this situation, I mean how to get the column name and those two values using a procedure which dynamically loop through row and check where the value is changing and then output those two values and the column name.

Help me ! if you provide me a code for this , it will be really helpful.

Sample dataset

SOURCE 
PK  COLUMN1 COLUMN2 COLUMN3 COLUMN4 
2   NAME2   VALUE2  3       4 
1   NAME1   VALUE1  2       3 
3   NAME3   VALUE3  4       5 

TARGET 
PK  COLUMN1 COLUMN2 COLUMN3 COLUMN4 
1   NAME1   VALUE1  2       3 
2   NAME2   VALUE2  4       4 
3   NAME3   VALUE3  4       5 

now

SELECT * FROM SOURCE MINUS SELECT * FROM TARGET 

gives

PK  COLUMN1 COLUMN2 COLUMN3 COLUMN4 

2   NAME2   VALUE2  3       4 

and

SELECT * FROM TARGET MINUS SELECT * FROM SOURCE 

gives

PK  COLUMN1 COLUMN2 COLUMN3 COLUMN4 

2   NAME2 VALUE2    4       4 

we can see that column3 value got changed from 3 to 4.

So what we need is something like this

COLUMN_NAME OLD_VALUE NEW_VALUE 

COLUMN3     3         4

Tables source and target are created from a procedure which take the sql for source and target table for another table that has two columns one is source query and other is target query and each row in this table has a different query for recon also the number of column and there name can change next time these table are created.

Answer 1

Assuming your temp1 and temp2 tables have same columns, its easy to do when you use EXECUTE IMMEDIATE, and know how to browse into Oracle system tables ALL_TABLES and ALL_TAB_COLUMNS.

Since I don't know how many columns temp tables have, the idea is to compare (with your original MINUS idea) the results of a concatenation of the columns. Beware you can't concatenate everything the same way (e.g. dates), so I showed how you could get DATA_TYPE.

Once you have the above result, you can manually see the column that changes. If I have time, I will add the part about the column that changed:

• if you have a PK, then we could use it to know the row who changed and loop again on the columns ;
• if no PK, it could get trickier...

I have much fun doing this, so I will try making a small code for it, assuming you PK is single column called PK:

create or replace procedure compare_tables(t1 in varchar2, t2 in varchar2)
is
    v_qry          varchar2(10000);
    TYPE T_MY_LIST IS TABLE OF VARCHAR2(32000);
    v_cols         T_MY_LIST;  -- list of columns
    v_types        T_MY_LIST;  -- list of columns' type
    v_cmp_cols     T_MY_LIST;  -- list of distinct
    v_col_t1_t2    T_MY_LIST;  -- t1 minus t2 - value of lines
    v_pk_t1_t2     T_MY_LIST;  -- associated PKs in t1 minus t2
    v_col_t2_t1    T_MY_LIST;  -- t2 minus t1 - value of lines
    v_pk_t2_t1     T_MY_LIST;  -- associated PKs in t2 minus t1
    TYPE T_Y_ IS TABLE OF VARCHAR2(32000) index by varchar2(1000);
    v_s                                            varchar2(1000); -- for indexing
    v_t1_t2        T_Y_; -- list of distinct lines from t1 - t2 /indexed by PK
    v_t2_t1        T_Y_; -- list of distinct lines from t2 - t1 /indexed by PK
begin
    -- the below assumes all tables have a PK called simply "PK".
    v_qry:='PK, ';
    execute immediate ' select COLUMN_NAME, DATA_TYPE '
                      ||' from ALL_TAB_COLUMNS where TABLE_NAME=upper('''||t1||''')' 
            bulk collect into v_cols, v_types;
    -- building query with list of columns:
    FOR I in 1..v_cols.count loop -- dbms_output.put_line(v_cols(i)||'.'||v_types(i));
        v_qry := v_qry||v_cols(i)||'||';
    end loop;
    v_qry := v_qry||'''''';
    execute immediate ' select '||v_qry||' from '||t1||' minus select '||v_qry||' from '||t2
            bulk collect into v_pk_t1_t2, v_col_t1_t2;
    execute immediate ' select '||v_qry||' from '||t2||' minus select '||v_qry||' from '||t1
            bulk collect into v_pk_t2_t1, v_col_t2_t1;

    -- build indexed structures that will help compare lines brought by "minus" queries
    FOR I in 1..v_pk_t1_t2.count loop
        v_t1_t2(v_pk_t1_t2(i)):=v_col_t1_t2(i);
    end loop;
    FOR I in 1..v_pk_t2_t1.count loop
        v_t2_t1(v_pk_t2_t1(i)):=v_col_t2_t1(i);
    end loop;

    v_s := v_t1_t2.FIRST;          -- Get first element of array
    WHILE v_s IS NOT NULL LOOP
        if (v_t2_t1.exists(v_s)) then
            -- distinct rows on same PK
            DBMS_Output.PUT_LINE (v_s || ' -> ' || v_t1_t2(v_s));

            -- loop on each column joined on PK:
            FOR i in 1..v_cols.count
            loop
                v_qry:= 'select '''||v_cols(i)||':''||'||t1||'.'||v_cols(i)||'||''<>''||'||t2||'.'||v_cols(i)
                      ||'  from '||t1||','||t2
                      ||' where '||t1||'.PK='||t2||'.PK'
                      ||'   and '||t1||'.PK='||v_s
                      ||'   and '||t1||'.'||v_cols(i)||'<>'||t2||'.'||v_cols(i)
                ;
                --DBMS_Output.PUT_LINE (v_qry);
                execute immediate v_qry bulk collect into v_cmp_cols;
                FOR j in 1..v_cmp_cols.count loop
                    DBMS_Output.PUT_LINE (v_cmp_cols(j));
                end loop;
            end loop;
        else 
            DBMS_Output.PUT_LINE (v_s || ' not in ' || t2);            
        end if;
      v_s := v_t1_t2.NEXT(v_s);    -- Get next element of array
    END LOOP;
    v_s := v_t2_t1.FIRST;          -- Get first
    WHILE v_s IS NOT NULL LOOP
        if (not v_t1_t2.exists(v_s)) then
            DBMS_Output.PUT_LINE (v_s || ' not in ' || t1);            
        end if;
      v_s := v_t2_t1.NEXT(v_s);    -- Get next
    END LOOP;
end compare_tables;
/

test data:

create table temp1 (PK number,
  COLUMN1 varchar2(10), 
  COLUMN2 varchar2(10),
  COLUMN3 varchar2(10),
  COLUMN4 varchar2(10)
  );

create table temp2 (PK number,
  COLUMN1 varchar2(10), 
  COLUMN2 varchar2(10),
  COLUMN3 varchar2(10),
  COLUMN4 varchar2(10)
  );
delete temp1;
insert into temp1 
          select 1, 'a', 'a', 'bb', 'cc' from dual
union all select 2, 'a', 'a', 'bb', 'cc' from dual
union all select 3, 'a', 'a', 'bb', 'cc' from dual
union all select 4, 'a', 'a', 'bb', 'cc' from dual
;
insert into temp2 
          select 1, 'a', 'a', 'bb', 'cc' from dual
union all select 2, 'a', 'a', 'b', 'cc'  from dual
union all select 3, 'a', 'a', 'bb', 'cc' from dual
;


begin
    compare_tables('temp1','temp2');
end;
/

Result:

2 -> 2aabbcc
COLUMN3:bb<>b
4 not in temp2

This was inspired by Search All Fields In All Tables For A Specific Value (Oracle) where the basic techinque is explained.