How to write to a CSV line by line?

Question

I have data which is being accessed via http request and is sent back by the server in a comma separated format, I have the following code :

site= 'www.example.com'
hdr = {'User-Agent': 'Mozilla/5.0'}
req = urllib2.Request(site,headers=hdr)
page = urllib2.urlopen(req)
soup = BeautifulSoup(page)
soup = soup.get_text()
text=str(soup)

The content of text is as follows:

april,2,5,7
may,3,5,8
june,4,7,3
july,5,6,9

How can I save this data into a CSV file. I know I can do something along the lines of the following to iterate line by line:

import StringIO
s = StringIO.StringIO(text)
for line in s:

But i'm unsure how to now properly write each line to CSV

EDIT---> Thanks for the feedback as suggested the solution was rather simple and can be seen below.

Solution:

import StringIO
s = StringIO.StringIO(text)
with open('fileName.csv', 'w') as f:
    for line in s:
        f.write(line)

It's already a CSV, you just have to write each line to a file... — icedwater, May 18 '16 at 03:51
I'm not sure you even need the `StringIO` import, to be honest. Also, the solution as-is probably does not separate the lines, as `f.write()` does not append newlines automatically. — icedwater, May 18 '16 at 07:48
@icedwater I understand what your saying, but I ran the code above and it was able to properly store the data to a csv file. — Mustard Tiger, May 18 '16 at 16:10
See also: [How do I read and write CSV files with Python?](https://stackoverflow.com/questions/41585078/how-do-i-read-and-write-csv-files-with-python/41585079#41585079) — Martin Thoma, Jan 05 '18 at 09:37

score 234 · Accepted Answer · edited Nov 27 '18 at 15:50

234

General way:

##text=List of strings to be written to file
with open('csvfile.csv','wb') as file:
    for line in text:
        file.write(line)
        file.write('\n')

OR

Using CSV writer :

import csv
with open(<path to output_csv>, "wb") as csv_file:
        writer = csv.writer(csv_file, delimiter=',')
        for line in data:
            writer.writerow(line)

OR

Simplest way:

f = open('csvfile.csv','w')
f.write('hi there\n') #Give your csv text here.
## Python will convert \n to os.linesep
f.close()

edited Nov 27 '18 at 15:50

gsamaras

71,951
46
188
305

answered May 18 '16 at 04:05

Ani Menon

27,209
16
105
126

25

for python 3, change it to `with open(, "w", newline='') as csv_file:` – Khaled Hamed Jul 18 '18 at 10:33
3

Missing information on this line `for line in data:`. Please fix that. Thank you. – Francisco Maria Calisto Oct 01 '18 at 19:38
1

how to append a line, if there is already something in the csv-file, using the third solution? – Jürgen K. Sep 12 '19 at 13:32
7

@JürgenK. Use `'a'`(append mode) in place of `'w'`(write mode). – Ani Menon Sep 19 '19 at 04:12
If you're getting the error: `TypeError: a bytes-like object is required, not 'str'`, just change the `'wb'` to `'w'` in the `open()` function. – tsveti_iko Sep 29 '20 at 13:09

score 20 · Answer 2 · answered May 18 '16 at 03:59

You could just write to the file as you would write any normal file.

with open('csvfile.csv','wb') as file:
    for l in text:
        file.write(l)
        file.write('\n')

If just in case, it is a list of lists, you could directly use built-in csv module

import csv

with open("csvfile.csv", "wb") as file:
    writer = csv.writer(file)
    writer.writerows(text)

icedwater · Answer 3 · 2021-10-05T02:48:03.923

13

I would simply write each line to a file, since it's already in a CSV format:

write_file = "output.csv"
with open(write_file, "wt", encoding="utf-8") as output:
    for line in text:
        output.write(line + '\n')

I can't recall how to write lines with line-breaks at the moment, though :p

Also, you might like to take a look at this answer about write(), writelines(), and '\n'.

edited Oct 05 '21 at 02:48

answered May 18 '16 at 03:55

icedwater

4,701
3
35
50

1

also care about encoding like this `open(write_file, 'wt', encoding='utf-8')` – Bellash Sep 28 '21 at 19:44

score 9 · Answer 4 · edited Mar 26 '23 at 17:39

In addition to the previous answers, I created a class for quickly writing to CSV files. This approach simplifies the management and closure of open files, as well as ensures consistency and cleaner code, especially when dealing with multiple files.

class CSVWriter():

    filename = None
    fp = None
    writer = None

    def __init__(self, filename):
        self.filename = filename
        self.fp = open(self.filename, 'w', encoding='utf8')
        self.writer = csv.writer(self.fp, delimiter=';', quotechar='"', quoting=csv.QUOTE_ALL, lineterminator='\n')

    def close(self):
        self.fp.close()

    def write(self, *args):
        self.writer.writerow(args)

    def size(self):
        return os.path.getsize(self.filename)

    def fname(self):
        return self.filename

Example usage:

mycsv = CSVWriter('/tmp/test.csv')
mycsv.write(12,'green','apples')
mycsv.write(7,'yellow','bananas')
mycsv.close()
print("Written %d bytes to %s" % (mycsv.size(), mycsv.fname()))

Have fun

score 2 · Answer 5 · answered Sep 20 '18 at 20:43

2

What about this:

with open("your_csv_file.csv", "w") as f:
    f.write("\n".join(text))

str.join() Return a string which is the concatenation of the strings in iterable. The separator between elements is the string providing this method.

answered Sep 20 '18 at 20:43

Vlad Bezden

83,883
25
248
179

score 0 · Answer 6 · answered Sep 12 '22 at 13:07

In my situation...

  with open('UPRN.csv', 'w', newline='') as out_file:
    writer = csv.writer(out_file)
    writer.writerow(('Name', 'UPRN','ADMIN_AREA','TOWN','STREET','NAME_NUMBER'))
    writer.writerows(lines)

you need to include the newline option in the open attribute and it will work

https://www.programiz.com/python-programming/writing-csv-files

How to write to a CSV line by line?

6 Answers6

Linked

Related