Reading csv from FTP folder

Question

I am trying to read csv file from FTP Folder

ftp = FTP('adr')  
ftp.login(user='xxxx', passwd = 'xxxxx')
r = StringIO()
ftp.retrbinary('RETR /DataLoadFolder/xxx/xxx/xxx/'+str(file_name),r.write)
r.seek(0)
csvfile1 = csv.reader(r,delimiter=';')
input_file = [list(line) for line in csv.reader(r)] ----- Error

getting an error at last line as

new-line character seen in unquoted field - do you need to open the file in universal-newline mode?

My csv file

Text Version

There are whites spaces at the end of each row (after 17.00)

Data starts from second row

what does the error mean? Any help would be much appreciated.

A csv file is **not** a spreadsheet but plain text. Once you load it into Excel, what you see in not the original file but the way it has been transformed. Please show the text version... — Serge Ballesta, Jun 07 '18 at 11:34
@SergeBallesta edited question and added text version. FYI there are around 70 white spaces at the end of each row — nick, Jun 12 '18 at 14:05
How can you feed a `StringIO` with bytes retrieved from `retrbinary`? Is this really your real code? — Serge Ballesta, Jun 12 '18 at 16:03

score 1 · Answer 1 · answered Jun 07 '18 at 12:01

1

The error message simply asking how you'd want to handle the newline differently due to historical reasons, you can read the explanation here.

To solve the issue, specify the newline on StringIO like this:

r = StringIO(newline='')

According to StringIO documentation. If newline is set to None, newlines are written as \n on all platforms, but universal newline decoding is still performed when reading.

answered Jun 07 '18 at 12:01

hcheung

3,377
3
11
23

nothing seems to be worked. please find edited question. Thanks – nick Jun 12 '18 at 14:06
Maybe you should test the code by reading the csv directly from file to make sure whether the StringIO/FTP reading cause the problem or the problem was caused by malformed csv file. – hcheung Jun 13 '18 at 00:24
problem was caused by malformed csv file, previously i used to read csv firectly from file with same code. – nick Jun 13 '18 at 05:41

score 1 · Answer 2 · answered Jun 12 '18 at 16:54

I could partially reproduce and fix. The error is caused by a line containing a bad end of line. I could reproduce by adding a line \r \n at the end of an otherway valid csv file.

A simple way to fix it is to use a filter to eliminate blank lines and clean end of lines:

def filter_bytes(fd):
    for line in fd:
        line = line.strip()
        if len(line) != 0:
            yield(line + b'\r\n')

Once this is done, your code could become:

ftp = FTP('adr')  
ftp.login(user='xxxx', passwd = 'xxxxx')
r = BytesIO()
ftp.retrbinary('RETR /DataLoadFolder/xxx/xxx/xxx/'+str(file_name),r.write)
r.seek(0)
csvfile1 = csv.reader(filter_bytes(r),delimiter=';')
input_file = list(csvfile1)

Reading csv from FTP folder

2 Answers2