0

I have the following code:

output = io.BytesIO()
some_function(output)   # some_function writes output to file n times
buffer = output.getbuffer()
output.getvalue()

output.getvalue() returns the following piece: b'\x01\x02\x03\x01\x02\x03\x01\x02\x03\x01

but imagine if this were a long piece.
the problem is with the get value() function that could potentially return a large number of data based on number of times some_function() is called. lets say n = 1000000. How would i process the binary data returned from get value() in chunks so that i can 

write(chunk) for each chunk in get value

to the file?

turtle_in_mind
  • 986
  • 1
  • 18
  • 36
  • its not a duplicate. its how to read binary data from memory in chunks. once i have a large memory in get value(), how would i store that chunk by chunk? the is a sample piece returned by getvalue() b'\x01\x02\x03\x01\x02\x03\x01\x02\x03\x01 – turtle_in_mind Sep 16 '19 at 21:53
  • If your function already generates the whole thing in memory, then why not write it out as one block? You've already taken the memory hit at this point anyway. If your function can be changed to generate it "lazily", the solution in the duplicate could apply. – joanis Sep 17 '19 at 02:28

0 Answers0