A stream implementation using an in-memory bytes buffer.
Refer to BytesIO class details in Python's docs.
Questions tagged [bytesio]
258 questions
116
votes
3 answers
Writing then reading in-memory bytes (BytesIO) gives a blank result
I wanted to try out the python BytesIO class.
As an experiment I tried writing to a zip file in memory, and then reading the bytes back out of that zip file. So instead of passing in a file-object to gzip, I pass in a BytesIO object. Here is the…
twasbrillig
- 17,084
- 9
- 43
- 67
75
votes
4 answers
Convert from '_io.BytesIO' to a bytes-like object in python3.6?
I am using this function to uncompress the body of a HTTP response if it is compressed with gzip, compress or deflate.
def uncompress_body(self, compression_type, body):
if compression_type == 'gzip' or compression_type == 'compress':
…
Dan
- 2,209
- 3
- 23
- 44
47
votes
7 answers
PIL cannot identify image file for io.BytesIO object
I am using the Pillow fork of PIL and keep receiving the error
OSError: cannot identify image file <_io.BytesIO object at 0x103a47468>
when trying to open an image. I am using virtualenv with python 3.4 and no installation of PIL.
I have tried to…
Elan M
- 471
- 1
- 4
- 5
39
votes
3 answers
How the write(), read() and getvalue() methods of Python io.BytesIO work?
I'm trying to understand the write() and read() methods of io.BytesIO.
My understanding was that I could use the io.BytesIO as I would use a File
object.
import io
in_memory = io.BytesIO(b'hello')
print( in_memory.read() )
The above code will…
Robert
- 31,925
- 8
- 35
- 33
32
votes
1 answer
Confusing about StringIO, cStringIO and ByteIO
I have googled and also search on SO for the difference between these buffer modules. However, I still don't understand very well and I think some of the posts I read are out of date.
In Python 2.7.11, I downloaded a binary file of a specific format…
wsdzbm
- 3,096
- 3
- 25
- 28
26
votes
1 answer
Convert file into BytesIO object using python
I have a file and want to convert it into BytesIO object so that it can be stored in database's varbinary column.
Please can anyone help me convert it using python.
Below is my code:
f = open(filepath, "rb")
print(f.read())
myBytesIO =…
user2961127
- 963
- 2
- 17
- 29
26
votes
1 answer
Writing to io.BytesIO in csv fails in python3
I am trying to write python 2/3 compatible code to write strings to csv file object. This code:
line_as_list = [line.encode() for line in line_as_list]
writer_file = io.BytesIO()
writer = csv.writer(writer_file, dialect=dialect,…
goelakash
- 2,502
- 4
- 40
- 56
24
votes
2 answers
Writing a BytesIO object to a file, 'efficiently'
So a quick way to write a BytesIO object to a file would be to just use:
with open('myfile.ext', 'wb') as f:
f.write(myBytesIOObj.getvalue())
myBytesIOObj.close()
However, if I wanted to iterate over the myBytesIOObj as opposed to writing it in…
Kalabaaz
- 422
- 1
- 5
- 7
22
votes
3 answers
Convert Pandas DataFrame to bytes-like object
Hi I am trying to convert my df to binary and store it in a variable.
my_df:
df = pd.DataFrame({'A':[1,2,3],'B':[4,5,6]})
my code:
import io
towrite = io.BytesIO()
df.to_excel(towrite) # write to BytesIO buffer
towrite.seek(0) # reset…
Pyd
- 6,017
- 18
- 52
- 109
17
votes
2 answers
When should one use BytesIO .getvalue() instead of .getbuffer()?
According to the BytesIO docs:
getbuffer()
Return a readable and writable view over the contents of the buffer without copying them. Also, mutating the view will transparently update the contents of the buffer:
getvalue()
Return bytes containing…
Martin Thoma
- 124,992
- 159
- 614
- 958
15
votes
2 answers
Can I pipe a io.BytesIO() stream to subprocess.popen() in Python?
I'm trying to pipe a io.BytesIO() bytetream to a separate program using subprocess.popen(), but I don't know how or if this is at all possible. Documentation and examples are all about text and newlines.
When I whip up something like this:
import…
Redsandro
- 11,060
- 13
- 76
- 106
13
votes
4 answers
Strange "BadZipfile: Bad CRC-32" problem
This code is simplification of code in a Django app that receives an uploaded zip file via HTTP multi-part POST and does read-only processing of the data inside:
#!/usr/bin/env python
import csv, sys, StringIO, traceback, zipfile
try:
import…
Marc Abramowitz
- 3,447
- 3
- 24
- 30
12
votes
1 answer
ftp sending python bytesio stream
I want to send a file with python ftplib, from one ftp site to another, so to avoid file read/write processees.
I create a BytesIO stream:
myfile=BytesIO()
And i succesfully retrieve a image file from ftp site one with…
jose
- 399
- 2
- 14
10
votes
1 answer
pandas read_csv from BytesIO
I have a BytesIO file-like object, containing a CSV.
I want to read it into a Pandas dataframe, without writing to disk in between.
MWE
In my use case I downloaded the file straight into BytesIO.
For this MWE I'll have a file on disk, read it into…
falsePockets
- 3,826
- 4
- 18
- 37
9
votes
2 answers
Why is TextIOWrapper closing the given BytesIO stream?
If I run following code in python 3
from io import BytesIO
import csv
from io import TextIOWrapper
def fill_into_stringio(input_io):
writer = csv.DictWriter(TextIOWrapper(input_io, encoding='utf-8'),fieldnames=['ids'])
for i in…
Yannick Widmer
- 1,286
- 3
- 19
- 30