0

I am trying to fetch data from the internet to save it to a csv file.
I am facing a problem with writing to the csv file, the library leaves an empty row in the file

Example

The data is random.org integers in text/plain format.

I'm using urllib.request to fetch the data and I am using this code to get the data and decode it 
req = urllib.request.Request(url, data=None, headers={
    'User-Agent': '(some text)'})

with urllib.request.urlopen(req) as response:
    html = response.read()
    encoding = response.headers.get_content_charset('utf-8')
    data = html.decode(encoding)

I am using this line of code to open the csv file :csvfile = open('data.csv', "a")
Writing to the file:
writer = csv.writer(csvfile, lineterminator = '\n') writer.writerows(data)
and of course I close the file at the end

Things I tried and didn't help :
  • Using (lineterminator = '\n') when writing
  • Using (newline = "") when opening the file
  • Defining a "delimiter" and a "quotechar" and "quoting"
Stephen Rauch
  • 47,830
  • 31
  • 106
  • 135
Oliver Wright
  • 5
  • 4

1 Answers1

1

Updated Answer

If when you create the list that is being written to the data list, if you add it as a list so that data becomes a list of lists, then set your line delimiter to be '\n', it should work. Below is the working code I used to test.

import csv
import random

csvfile = open('csvTest.csv', 'a')
data = []
for x in range(5):
    data.append([str(random.randint(0, 100))])

writer = csv.writer(csvfile, lineterminator = '\n')
writer.writerows(data)
csvfile.close()

and it outputs

enter image description here

njoosse
  • 549
  • 3
  • 8
  • as an additional note, this was done with the default line terminator. – njoosse May 25 '17 at 20:17
  • I was using binary mode, then I got into problems of converting str to byte(either an error or writing wrong data like 48 and 49), any help ? – Oliver Wright May 25 '17 at 20:21
  • you can set the dialect to `excel-tab` which will cause the numbers > 9 to be separated by a tab but they will appear correctly in Excel. `writer = csv.writer(csvfile, dialect = 'excel-tab')` – njoosse May 25 '17 at 20:39
  • Also, using binary mode didn't help unfortunately. – Oliver Wright May 25 '17 at 21:46
  • This works but it keeps using the same value that is getting fetched the first time like : (0,0,0,0,0), I want to figure it out by myself. Thank you <3 – Oliver Wright May 26 '17 at 00:32