2

I am pulling in a csv string and need to parse it into a pd dataframe. Example string:

b'date,"total revenue"\n2018-06-19,12.65\n2018-06-20,3.90\n2018-06-21,6.16\n2018-06-22,9.06\n2018-06-23,1.30\n2018-06-24,1.88\n2018-06-25,4.20\n2018-06-26,2.46\n2018-06-27,2.38\n2018-06-28,1.06\n`

How can I convert this string into a 2 column pandas dataframe? Many thanks

I tried the following:

df = pd.DataFrame(list(reader(str(my_string))))

But the output is all skewed:

                   0     1
0                  b  None
1                  '  None
2                  d  None
3                  a  None
4                  t  None
5                  e  None
6                         
7      total revenue  None
8                  \  None
Laurence_jj
  • 646
  • 1
  • 10
  • 23

4 Answers4

3

I found the answer here (How to convert bytes data into a python pandas dataframe?):

from io import StringIO

s=str(bytes_data,'utf-8')

data = StringIO(s) 

df=pd.read_csv(data)
Laurence_jj
  • 646
  • 1
  • 10
  • 23
1
import sys
import pandas as pd

if sys.version_info[0] < 3: 
    from StringIO import StringIO
else:
    from io import StringIO


temp =  StringIO(my_string)
df =pd.read_csv(temp, sep=",", lineterminator='\n')

df

You migth try this, and dont forget to remove b as it take rows for df as binary

Manish Chaudhary
  • 498
  • 6
  • 14
1

I use this way if the binary data is a csv file.

with open("test.csv", 'wb') as f:  
    f.write(bytes_data)
df = pd.read_csv('test.csv')
ERIC
  • 460
  • 6
  • 16
1

You can simply use io.BytesIO instead of converting to string and using io.StringIO :
For example, retrieving CSV data from an API :

import io
import requests
r = requests.get("http://127.0.0.1:5000/get_logs")
data = io.BytesIO(r.content)
df = pd.read_csv(data)

(This is what Eric proposed above, but without the need to actually save the data to a persistent static file)

Durand
  • 67
  • 6