I have a custom function written to fetch data from csv files which does not work more than once in the code

Question

I have a csv file with 2 columns and 11 rows only (one header row and 10 number rows).

I use custom function to fetch data from the csv file using csv library. The csv file outputs column1 or column2 as a list based on user's choice. The input parameters are:

name_string : it is the name of the column the user desires
data_file : it is the csv file which is opened by using csv library

The function is:

def fetch(name_string, data_file):
    x = 0
    location = 0
    output = []

    if name_string == "ONE":
        location = 0
    elif name_string == "TWO":
        location = 1
    else:
        return print("An unknown error occurred")

    for line in data_file:                          
        if x > 0:
            output.append(float(line[location]))              
        x += 1

    return output

While using the function inside the code, the function works just fine but only once. On second time it returns an empty list []. I still can't understand why this is happening?

The code is:

import csv

def fetch(name_string, data_file):
    x = 0
    location = 0
    output = []

    if name_string == "ONE":
        location = 0
    elif name_string == "TWO":
        location = 1
    else:
        return print("An unknown error occurred")

    for line in data_file:                          
        if x > 0:
            output.append(float(line[location]))              
        x += 1

    return output


with open('test_file.csv', 'r') as data1:

    rawData = csv.reader(data1)

    Col1 = fetch("ONE", rawData)
    Col2 = fetch("TWO", rawData)

    print(Col1)
    print(Col2)

Col1 outputs just fine but Col2 outputs as [].

The csv file is available here

if you only do col2, what happens? if you use the argument "ONE" the second time you call fetch, what happens? — JoshuaF, Jan 03 '20 at 20:32
[csv_reader returns an object that implements the iterator protocol](https://docs.python.org/3/library/csv.html). So as an iterator, it can only be traversed once. Consequently, you may want to pass fetch the filename and had it read the file. — DarrylG, Jan 03 '20 at 20:35
Same thing, the function fetches data only the first time. It returns a blank list the second time. — Asad Ullah Butt, Jan 03 '20 at 20:36
You are consuming the data that you have read which is `rawData`. — BcK, Jan 03 '20 at 20:36
Your way of skipping the first line of the document is a bit dirty. Maybe consider using `for n, line in enumerate(data_file): if n > 0: output.append(...)`. — Guimoute, Jan 04 '20 at 20:00

score 1 · Answer 1 · answered Jan 03 '20 at 20:39

So, I haven't tried this...but, I believe the way that library works is by opening like a filestream...

So, on the first iteration you are reading until the EOF (End Of File) which I would guess is why on the second call of fetch the "filestream" object has nothing to return since it already read the whole file and outputs [].

One way to do solve your code easily is to open the file inside of the "fetch" function...although this is not efficient at all.

Another way would be to store the file in memory in a structure you think appropriate (dict, list, etc). And then, perform your printing of the data based on that object.

Hope this guess helps a bit...

score 1 · Answer 2 · answered Jan 03 '20 at 20:43

I think this is because your traversing the csv.reader object once in your first call of the function. And when you call the function again, there is nothing to read because you're already at the end of your CSV.

One easy fix in for your approach is to pass the csv filename to your function, and do the file opening and reading inside the function.

score 1 · Answer 3 · answered Jan 03 '20 at 20:48

1

csv.reader() returns an iterator, and iterators can only be iterated through once.

You may wish to either convert the csv contents to a list, or rework your function to process both columns at the same time.

answered Jan 03 '20 at 20:48

Phil

13
4

score 1 · Answer 4 · answered Jan 03 '20 at 20:59

You should definitely read the csv into the memory and then do whatever you need.

By the way please search before asking a question.

How to use csv reader object multiple times

import csv

def fetch(name_string, data):
    x = 0
    location = 0
    output = []

    if name_string == "ONE":
        location = 0
    elif name_string == "TWO":
        location = 1
    else:
        return print("An unknown error occurred")

    for line in data:
        if x > 0:
            output.append(float(line[location]))              
        x += 1

    return output


with open("test_file.csv", "r") as f:
    reader = csv.reader(f)
    data = list(reader)

col1 = fetch("ONE", data)
col2 = fetch("TWO", data)

print(col1)
print(col2)

I have a custom function written to fetch data from csv files which does not work more than once in the code

4 Answers4