Continuously write data from a list into a CSV file

Question

I'm trying to get my python script to keep writing to a .CSV file and I'm quite the beginner when it comes to python.

I've tried to follow this person which has a similar problem but without success: Writing List of Strings to Excel CSV File in Python

I'm currently trying to save a list of sensor data containing temperature, humidity and a time, the code currently look like this:

def writeDataToFile():
    sensorData = []
    sensorData = getSensorData()


    with open("/home/pi/Documents/Python Projects/CSV-files/sensor_data.csv", 'w') as csv_file:
        writer = csv.writer(csv_file, delimiter=',')
        writer.writerows(sensorData)

The error i'm getting is this: interable expected, not int So the first data values is stored in a int i assume?

I'm used to java where you define types but here you just type the variable so uhm yeah.

I've tried writer.writerow(sensorData) and it works then, but as you might already expect, it just writes a single row of data.

The rest of the code is running in a while true loop where it keeps storing data into the list.

How do i get this function to keep writing to my csv file, adding more and more as my loop keeps running?

The getSensorData function:

def getSensorData():
    sensorData = []

    temperature = getTemperatureFromSensor()
    humidity = getHumidityFromSensor()
    time = getCurrentFormatedTimestamp()

    sensorData.append(temperature)
    sensorData.append(humidity)
    sensorData.append(time)

    return sensorData

So i tried to simply print out the list and it does exactly what i want it do to:

[29, 21, '2017-10-30 15:02:47']

[29, 21, '2017-10-30 15:02:52']

[29, 22, '2017-10-30 15:02:57']

[29, 21, '2017-10-30 15:03:02']

[28, 21, '2017-10-30 15:03:07']

[28, 21, '2017-10-30 15:03:13']

etc, that's basically what i want to store into the csv

IIUC, the issue is that you're opening the file in write mode `w` which completely erases all existing contents. You should open in append mode. It appears that `getSensorData()` will return an `int`, so for each time you call that method, you'll need to `writerow` that value. — roganjosh, Oct 30 '17 at 13:49
Ok, so `writerows` expects you to pass a nested list, where each inner list represents a row. If you pass a single list, it will find an int value at index 0 while it actually is expecting a list representing the first row. That's where the error comes from. You should be using `writerow`. That's in addition to my previous comment; you need append mode to not overwrite the previous data each time you open the file. — roganjosh, Oct 30 '17 at 13:57

score 3 · Accepted Answer · answered Oct 30 '17 at 14:05

3

The writerows method of the csv module expects a nested list where each inner list represents the data for a single row. In your case, passing a flat list, it instead finds an int value at the 0th index where it's instead expecting the inner list for the first row. That explains the error message.

So, you should instead be using writerow each time you call your method. However, opening the file in write mode ('w') will delete all the contents of the file each time. Therefore, you need to open the file in append mode. A toy example:

import csv

def write_csv(data):
    with open('example.csv', 'a') as outfile:
        writer = csv.writer(outfile)
        writer.writerow(data)

for x in range(5):
    write_csv([x, x+1])

answered Oct 30 '17 at 14:05

roganjosh

12,594
4
29
46

I feel so stupid.. haha, i guess i need to and study abit more python. – Patt089 Oct 30 '17 at 14:37
@Patt089 no worries, it will come with time. Note that in Python we generally don't use camel case like Java, function names are lower-case and separated by underscores; see [PEP8](https://www.python.org/dev/peps/pep-0008/#prescriptive-naming-conventions). If you feel that this answer has solved your issue, I would be grateful if you could [mark it as correct](https://meta.stackexchange.com/questions/5234/how-does-accepting-an-answer-work). – roganjosh Oct 30 '17 at 14:43
I'm trying to modify it over your example. Now i'm getting that my method takes 1 argument and i've given 0. But it's something minor i bet, i'll look it up and come back here and mark it as correct once i manage to solve this new error – Patt089 Oct 30 '17 at 14:52
@Patt089 What method is it complaining about? The only change you need to make based on my answer should be to change `writerows` to `writerow` and change `w` to `a`. – roganjosh Oct 30 '17 at 14:53
@Patt089 have you created an argument called `data` on one of your functions i.e. `def writeDataToFile(data)` or `def getSensorData(data):`? There is no need for you to do that, it's just part of my example that shows the general principle of appending to a CSV being continually opened. – roganjosh Oct 30 '17 at 15:01
Oh i took the later part of your code too, where you made a iteration through write_csv, then i realized that i'm not passing down anything through my parameters so i removed it and... It WORKS! Thanks alot rogan! – Patt089 Oct 30 '17 at 15:04
How do i mark this as a correct answer? I thought it was simpy done by upvoting your answer haha. Edit: Found it! – Patt089 Oct 30 '17 at 15:07
@Patt089 You click the tick next to my answer. See the link in my first comment: https://meta.stackexchange.com/questions/5234/how-does-accepting-an-answer-work – roganjosh Oct 30 '17 at 15:07

Continuously write data from a list into a CSV file

1 Answers1