Open and read all binary files from directory an output to lists - Python

Question

I'm trying to output the entire contents of multiple .pkt files, containing binary data, as a series of hexadecimal lists: (Adapted from How to open every file in a folder)

import glob
path = 'filepath'
for filename in glob.glob(os.path.join(path, '*.pkt')):    
   with open(os.path.join(os.getcwd(), filename), 'rb') as f:
   pair_hex = ["{:02x}".format(c) for c in f.read()]
   print(pair_hex)

Which outputs:

['09', '04', '04', '04', '04', '04', '04', '04', '04', '0b', '09']
['09', '04', 'bb']
['09', 'bb']
['bb']
['09', '04', '0b', '09']

That makes sense, because i'm looping through the files, but what I need is:

[['09', '04', '04', '04', '04', '04', '04', '04', '04', '0b', '09'],['09', '04', 'bb'],['09', 'bb'],['bb'],['09', '04', '0b', '09']]

So I can manipulate all the data.

I have tried to apply append(), "".join(), map() as well as the advice at How to merge multiple lists into one list in python?, but nothing changes the output. How can I get the desired list of lists?

Put the code in a function, and then apply the advice needed to `return` the list of lists after creating it in the loop. Alternately, use a *list comprehension* to process the files. — Karl Knechtel, Sep 06 '22 at 06:30

score 1 · Accepted Answer · 2020-06-23T19:35:01.507

1

untested but try

import glob, os
path = 'filepath'
ret = []
for filename in glob.glob(os.path.join(path, '*.pkt')):    
    with open(os.path.join(os.getcwd(), filename), 'rb') as f:
        pair_hex = ["{:02x}".format(c) for c in f.read()]
        ret.append(pair_hex)

print(ret)

the above prints the following on my console which is the same as your "desired output"

[['09', '04', '04', '04', '04', '04', '04', '04', '04', '0b', '09'], ['09', '04', 'bb'], ['09', 'bb'], ['bb'], ['09', '04', '0b', '09']]

and this is what I used to create the .pkt files on my machine with out set to a copy--paste of your "desired output"

out = [['09', '04', '04', '04', '04', '04', '04', '04', '04', '0b', '09'],['09', '04', 'bb'],['09', 'bb'],['bb'],['09', '04', '0b', '09']]

for i, a in enumerate(out):
    with open(f"{i}.pkt", 'w') as f:
        f.write(''.join(map(lambda s: chr(int(s, 16)), a)))

edited Jun 23 '20 at 19:35

answered Jun 23 '20 at 18:45

This leads to a loop of where each list size is multiplied by the number of files, which is 19. So it leads to 1*2*3*...*19. I tried to place ret [] within the loop, but this results in the same as above. – red_sach Jun 23 '20 at 18:59
This only nests the list further [['09','03']] [['01',02']] etc. They're still not all in the same list i.e. [['09','03'], ['01',02']] – red_sach Jun 23 '20 at 19:12
Then I give up; my earlier answer had it all in one list but you said that's not what you wanted. – Jun 23 '20 at 19:17
No sorry, it looped! x1 ['09', '04', '04', '04', '04', '04', '04', '04', '04', '0b', '09'] x2 ['09', '04', '04', '04', '04', '04', '04', '04', '04', '0b', '09'] ['09', '04', 'bb'] x3['09', '04', '04', '04', '04', '04', '04', '04', '04', '0b', '09'] ['09', '04', 'bb'] ['09', 'bb'] x4['09', '04', '04', '04', '04', '04', '04', '04', '04', '0b', '09'] ['09', '04', 'bb'] ['09', 'bb'] ['bb'] .I only want each list displayed once, as shown above? – red_sach Jun 23 '20 at 19:22
brought it back to my original answer and after creating some .pkt files, this prints the "desired output" in your question – Jun 23 '20 at 19:30
Correct you are! No sure what happened with my first implementation, but thanks a lot Justin. – red_sach Jun 24 '20 at 09:50

Open and read all binary files from directory an output to lists - Python

1 Answers1