How to convert hex data to byte data in ECL?

Question

I am working on a code in ECL and trying to convert hex data into byte data. I am using the following line of code for this:

hex_data := Std.Str.ToHexPairs((>DATA<)ds.contents);
d1:=Std.Str.FromHexPairs(hex_data);

But when I uncomment the second line, I get an error. Can someone please guide me on how to correctly convert hex data to byte data in ECL?

This is the entire code am working on:

IMPORT PYTHON3 AS PYTHON;
IMPORT Std;


// Note that the filesize doesn't really show up anywhere, even though we sprayed with FILENAME,FILESIZE.
// It is embedded in the length of the contents field.
myRec := RECORD
 STRING filename;
 DATA contents;
END;
// I had sprayed a list of files (2) to 'thor::img.flat'.  Make sure you run both the spray and the job on the thor cluster.
ds := DATASET('img.flat', myRec, THOR);
hex_data := Std.Str.ToHexPairs((>DATA<)ds.contents);
//d1:=Std.Str.FromHexPairs(hex_data);
// First output is just the file as we read it.
OUTPUT(ds[1]);
//OUTPUT(hex_data);
DATASET({UNSIGNED4 len, STRING filename, STRING img_bits, STRING img_byte}) pyProcess(STREAMED DATASET(myRec) recs) := EMBED(Python: Activity)
 import binascii
 for rec in recs:
   # Records come in as a tuple of fields
   name, dat = rec
   header = dat[:16]
   img_hex = binascii.hexlify(dat)
   img_bytes=binascii.unhexlify(img_hex)
 
   headerstr = ''.join(format(x, '02x') for x in header)
   # We yield one record at a time (as a tuple), or we could have
   # made a list and returned the whole list as one.  Using yield,
   # we can process as a stream, without having to load everythin
   # into memory at the same time.
   yield (len(dat), name, img_hex, img_bytes)


ENDEMBED;
// Second output is the results returned by python
OUTPUT(pyProcess(ds))

score 1 · Answer 1 · answered May 05 '23 at 13:16

OK, I just tested with this code:

IMPORT Std;
myRec := RECORD
  STRING filename;
  DATA contents;
END;
ds := DATASET([{'file1',D'123456ABCD'},{'file2',D'ABCD123456'}], myRec);
hex_data := Std.Str.ToHexPairs(ds[1].contents);
d1:=Std.Str.FromHexPairs(hex_data);
OUTPUT(ds);
OUTPUT(d1);
OUTPUT(hex_data);

You will note that my hex_data definition is different from yours, in two ways:

The type transfer operator (>DATA<) is unnecessary since the contents field is already defined as a DATA type.
I added the [1] reference to indicate the contents field from the first record, only. Without that, there would be a syntax error "no specified row for Table ds"

With those two changes, this code all runs correctly (including the d1 definition). I suggest you incorporate those changes and see where you are then.'

HTH,

Richard

Thanks, it is working with hard coded STRINGS as in ur code. But in my am trying to convert a BLOB sprayed image file to Bytes, `ds := DATASET('img.flat', myRec, THOR);` so is there any way I can convert the hex data returned by it as byte? — Dumbledore, May 06 '23 at 10:29
The image file can be processed by using a simple PROJECT. ImgStream := PROJECT(ds,TRANSFORM({STRING hex_data}, SELF.hex_data := Std.Str.ToHexPairs(LEFT.contents))); OUTPUT(ImgStream); — Bob Foreman, May 06 '23 at 16:43

How to convert hex data to byte data in ECL?

1 Answers1