Is there a way to understand what Oracle DataDump util updates in dmp file after extract?

Question

I do not want to wait for Oracle DataDump expdb to finish writing to dump file. So I start reading data from the moment it's created. Then I write this data to another file.

It worked ok - file sizes are the same (the one that OracleDump created and the one my data monitoring script created).

But when I run cmp it shows difference in 27 bytes:

cmp -l ora.dmp monitor_10k_rows.dmp

     3 263 154
     4 201 131
     5 174 173
     6 103  75
    48  64  70
    58   0 340
    64   0   1
    65   0 104
    66   0 110
   541  60  61
   545  60  61
   552  60  61
   559  60  61
 20508   0  15
 20509   0 157
 20510   0 230
 20526   0  10
 20532   0  15
 20533   0 225
 20534   0 150
913437   0 226
913438   0  37
913454   0  10
913460   0   1
913461   0 104
913462   0 100

 ls -al  ora.dmp
-rw-r--r-- 1 oracle oinstall 999424 Jun 20 11:35 ora.dmp

 python -c 'print 999424-913462'
 85962

 od ora.dmp -j 913461 -N 1
3370065 000100
3370066

 od monitor_10k_rows.dmp -j 913461 -N 1
3370065 000000
3370066

Even if I extract more data the difference is still 27 bytes but different addresses/values:

cmp -l ora.dmp monitor_30k_rows.dmp

      3 245 134
      4 222 264
      5 377 376
      6  54  45
     48  36  43
     57   0   2
     58   0 216
     64   0   1
     65   0 104
     66   0 120
    541  60  61
    545  60  61
    552  60  61
    559  60  61
  20508   0  50
  20509   0 126
  20510   0 173
  20526   0  10
  20532   0  50
  20533   0 174
  20534   0 120
2674717   0 226
2674718   0  47
2674734   0  10
2674740   0   1
2674741   0 104
2674742   0 110

Some writes are the same. Is there a way know addresses of bytes which will differ?

 ls -al  ora.dmp
-rw-r--r-- 1 bicadmin bic 2760704 Jun 20 11:09 ora.dmp

python -c 'print 2760704-2674742'
85962

How can update my monitored copy after DataDump updated the original at adress 2674742 using Python for example?

Exact same thing happens if I use COMPRESSION=DATA_ONLY option.

Update: Figured how to sync bytes that differ between 2 files:

def patch_file(fn, diff):
    for line in diff.split(os.linesep):
        if line:
            addr, to_octal, _ = line.strip().split()
            with open(fn , 'r+b') as f:
                f.seek(int(addr)-1)
                f.write(chr(int (to_octal,8)))

diff="""
     3 157 266
     4 232 276
     5 272 273
     6  16  25
    48  64  57
    58 340   0
    64   1   0
    65 104   0
    66 110   0
   541  61  60
   545  61  60
   552  61  60
   559  61  60
 20508  15   0
 20509 157   0
 20510 230   0
 20526  10   0
 20532  15   0
 20533 225   0
 20534 150   0
913437 226   0
913438  37   0
913454  10   0
913460   1   0
913461 104   0
913462 100   0
"""

patch_file(f3,diff)

Do you see the same sort of thing if you wait until the export has finished before copying the file? Maybe it's something to do with *how* you're copying it, rather than *when*? — Alex Poole, Jun 20 '17 at 17:37
I still suspect it's something to do with how you're reading and writing it, so showing that code might be helpful. I doubt it's going back and modifying what it's already written to file, although the first few at least look suggestive; I doubt you'd be able to 'fix' the file afterwards if it is though, since its a proprietary and private format. — Alex Poole, Jun 20 '17 at 17:49
also do youhave example how to path if I have to modify just one byte? — olekb, Jun 20 '17 at 17:52
I see, you want to use the `cmp` output to modify the new file. [Something like this](https://stackoverflow.com/q/4783657/266304)? Or maybe [this Python one](https://stackoverflow.com/q/508983/266304) would suit you better. — Alex Poole, Jun 20 '17 at 17:56
i tried this Python code. I'm struggling to translate cmp optput to anything meaningful. `2674742 0 110`. i fh.seek(2674742-1). then what is 110? Octal? what do i put in fh.write(???) — olekb, Jun 20 '17 at 18:03
According to the man page yes, it's octal; so you'd need to convert it to hex I suppose. ALso not sure you want the -1 in the offset, but this isn't my area *8-) — Alex Poole, Jun 20 '17 at 18:14

olekb · Answer 1 · 2017-06-28T12:28:31.177

wrote a patch using Python:

addr=[3 , 4 , 5 , 6 , 48 , 58 , 64 , 65 , 66 , 541 , 545 , 552 , 559 , 20508 , 20509 , 20510 , 20526 , 20532 , 20533 , 20534 ]
last_range=[85987, 85986, 85970, 85964, 85963, 85962]

def get_bytes(addr):
    out =[]
    with open(f1 , 'r+b') as f:
        for a in addr:
            f.seek(a-1)
            data= f.read(1)
            hex= binascii.hexlify(data)
            binary = int(hex, 16)
            octa= oct(binary)
            out.append((a,octa))

    return out


def patch_file(fn, bytes_to_update):
    with open(fn , 'r+b') as f:
        for (a,to_octal) in bytes_to_update:
            print (a,to_octal)
            f.seek(int(a)-1)
            f.write(chr(int (to_octal,8)))


if 1:
    from_file=f1
    fsize=os.stat(from_file).st_size
    bytes_to_read = addr + [fsize-x for x in last_range]

    bytes_to_update = get_bytes(bytes_to_read)  
    to_file =f3
    patch_file(to_file,bytes_to_update)

The reason I do dmp file monitoring is because it cuts backup time in half.

Is there a way to understand what Oracle DataDump util updates in dmp file after extract?

1 Answers1