Python: How to open files through Ftp with UTF-8/Unicode name?

Question

I want to connect to an FTP server and get files and iterate through items with python. but folder names are Persian/Arabic and when I want to open any folder it returns this :

error_perm: 550 The filename, directory name, or volume label syntax is incorrect.

these are things that I've tried:

from ftplib import FTP
import os
ftp = FTP(host_address)
ftp.login(user=user_name,passwd=password)
ftp.encoding='utf-8'
ftp.cwd("تست")

and

ftp = FTP(host_address,user_name,password)
ftp.encoding='utf-8'
ftp.cwd("تست")

and

with FTP(host_address) as ftp:
    ftp.login(user=user_name,passwd=password)
    ftp.encoding='utf-8'

and

import ftputil

with ftputil.FTPHost(host_address, user_name, password) as ftp_host:
    print (ftp_host.listdir(ftp_host.curdir))

...

the current directory in the server is something like this:

test
new folder
شیراز
تهران
تست

and if I get the list of items of the current directory through python it gives me something like this :

test 
new folder 
?????
?????
???

with these methods, I've used so far there is no problem to read files in test or new folder but not the other directories. how can I iterate through these folders or open them?

update: (more info)
more info with debug_level as snakecharmerb suggested in comments: I used this code (tried in ubuntu terminal and eclipse and Jupiter lab and ipython):

#!/usr/bin/env python3.6
# -*- coding: utf-8 -*-

from ftplib import FTP
import os
from six import u as unicode

host_address = "**********"
user_name = "***********"
password = "********"

ftp = FTP(host_address)
ftp.login(user=user_name,passwd=password)
ftp.set_debuglevel(2)
ftp.pwd()
for item in ftp.nlst():
    print(item)
    ftp.cwd(item)
    ftp.retrlines("LIST")
    ftp.cwd("../")

and this is the output :

*cmd* 'PWD'
*put* 'PWD\r\n'
*get* '257 "/" is current directory.\r\n'
*resp* '257 "/" is current directory.'
*cmd* 'TYPE A'
*put* 'TYPE A\r\n'
*get* '200 Type set to A.\r\n'
*resp* '200 Type set to A.'
*cmd* 'PASV'
*put* 'PASV\r\n'
*get* '227 Entering Passive Mode (192,168,22,3,20,18).\r\n'
*resp* '227 Entering Passive Mode (192,168,22,3,20,18).'
*cmd* 'NLST'
*put* 'NLST\r\n'
*get* '125 Data connection already open; Transfer starting.\r\n'
*resp* '125 Data connection already open; Transfer starting.'
*get* '226 Transfer complete.\r\n'
*resp* '226 Transfer complete.'
all file
*cmd* 'CWD all file'
*put* 'CWD all file\r\n'
*get* '250 CWD command successful.\r\n'
*resp* '250 CWD command successful.'
*cmd* 'TYPE A'
*put* 'TYPE A\r\n'
*get* '200 Type set to A.\r\n'
*resp* '200 Type set to A.'
*cmd* 'PASV'
*put* 'PASV\r\n'
*get* '227 Entering Passive Mode (192,168,22,3,20,19).\r\n'
*resp* '227 Entering Passive Mode (192,168,22,3,20,19).'
*cmd* 'LIST'
*put* 'LIST\r\n'
*get* '125 Data connection already open; Transfer starting.\r\n'
*resp* '125 Data connection already open; Transfer starting.'
11-21-18  11:12AM       <DIR>          CpayImageView Code dar Jadid
11-21-18  11:13AM       <DIR>          New folder
11-21-18  11:13AM       <DIR>          UltraSearch-x64
11-21-18  11:13AM       <DIR>          UltraSearch-x86
*get* '226 Transfer complete.\r\n'
*resp* '226 Transfer complete.'
*cmd* 'CWD ../'
*put* 'CWD ../\r\n'
*get* '250 CWD command successful.\r\n'
*resp* '250 CWD command successful.'
 New folder
*cmd* 'CWD New folder'
*put* 'CWD New folder\r\n'
*get* '250 CWD command successful.\r\n'
*resp* '250 CWD command successful.'
*cmd* 'TYPE A'
*put* 'TYPE A\r\n'
*get* '200 Type set to A.\r\n'
*resp* '200 Type set to A.'
*cmd* 'PASV'
*put* 'PASV\r\n'
*get* '227 Entering Passive Mode (192,168,22,3,20,20).\r\n'
*resp* '227 Entering Passive Mode (192,168,22,3,20,20).'
*cmd* 'LIST'
*put* 'LIST\r\n'
*get* '125 Data connection already open; Transfer starting.\r\n'
*resp* '125 Data connection already open; Transfer starting.'
*get* '226 Transfer complete.\r\n'
*resp* '226 Transfer complete.'
*cmd* 'CWD ../'
*put* 'CWD ../\r\n'
*get* '250 CWD command successful.\r\n'
*resp* '250 CWD command successful.'
p2512067424
*cmd* 'CWD p2512067424'
*put* 'CWD p2512067424\r\n'
*get* '250 CWD command successful.\r\n'
*resp* '250 CWD command successful.'
*cmd* 'TYPE A'
*put* 'TYPE A\r\n'
*get* '200 Type set to A.\r\n'
*resp* '200 Type set to A.'
*cmd* 'PASV'
*put* 'PASV\r\n'
*get* '227 Entering Passive Mode (192,168,22,3,20,21).\r\n'
*resp* '227 Entering Passive Mode (192,168,22,3,20,21).'
*cmd* 'LIST'
*put* 'LIST\r\n'
*get* '150 Opening ASCII mode data connection.\r\n'
*resp* '150 Opening ASCII mode data connection.'
 06-19-17  09:10AM       <DIR>          1396-03-03
 06-19-17  09:10AM       <DIR>          1396-03-04
06-19-17  09:10AM       <DIR>          1396-03-06
06-19-17  09:11AM       <DIR>          1396-03-07
06-19-17  09:11AM       <DIR>          1396-03-08
06-19-17  09:11AM       <DIR>          1396-03-09
06-19-17  09:11AM       <DIR>          1396-03-10
06-19-17  09:11AM       <DIR>          1396-03-11
06-19-17  09:12AM       <DIR>          1396-03-13
06-19-17  09:12AM       <DIR>          1396-03-16
06-19-17  09:12AM       <DIR>          1396-03-17
06-19-17  09:12AM       <DIR>          1396-03-18
*get* '226 Transfer complete.\r\n'
*resp* '226 Transfer complete.'
*cmd* 'CWD ../'
*put* 'CWD ../\r\n'
*get* '250 CWD command successful.\r\n'
*resp* '250 CWD command successful.'
?????
*cmd* 'CWD ?????'
*put* 'CWD ?????\r\n'
*get* '550 The filename, directory name, or volume label syntax is incorrect. \r\n'
*resp* '550 The filename, directory name, or volume label syntax is incorrect. '
Traceback (most recent call last):
  File "/home/h128/work_space3/ftppython/ftppythonteset.py", line 21, in <module>
    ftp.cwd(item)
  File "/usr/lib/python2.7/ftplib.py", line 574, in cwd
    return self.voidcmd(cmd)
  File "/usr/lib/python2.7/ftplib.py", line 256, in voidcmd
    return self.voidresp()
  File "/usr/lib/python2.7/ftplib.py", line 231, in voidresp
    resp = self.getresp()
  File "/usr/lib/python2.7/ftplib.py", line 226, in getresp
    raise error_perm, resp
ftplib.error_perm: 550 The filename, directory name, or volume label syntax is incorrect.

you see, while it is iterating through the folders with English name it has no problem, it can enter the folder, print the list of files in that folder and then come back to the upper folder. but when a folder with utf-8 name came up , it shows ???? and can not find the folder.

Maybe your stdout doesn't support utf-8 characters? What env are you on? — Ardein_, May 14 '19 at 13:23
@AdrianDaniszewski I tried on jupyterLab and ubuntu terminal python — hossein hayati, May 14 '19 at 13:25
Those `?` characters are generated when *ASCII* text is read using the wrong codepage. Incorrect conversions from ASCII to Unicode would generate the Unicode replacement character � instead — Panagiotis Kanavos, May 14 '19 at 13:26
@PanagiotisKanavos the problem is when I want to move to one of these directories I got the error. like there are in the code `ftp.cwd("تست")` — hossein hayati, May 14 '19 at 13:29
@PanagiotisKanavos so it means that it does not convert to `UTF-8` ? — hossein hayati, May 14 '19 at 13:47
Can you set the [debug level](https://docs.python.org/3/library/ftplib.html#ftplib.FTP.set_debuglevel) to see if that provides more information about what, exactly is being sent (and received)? It's very difficult to tell whether the problem is in the client or the server. — snakecharmerb, May 15 '19 at 07:35
Your code works for me in Python 3.6 with Arabic file names. So it's rather an interoperability problem between your FTP server and ftplib. What FTP server is that? Show us a complete debug log (including the initial exchange). — Martin Prikryl, May 20 '19 at 06:07
@MartinPrikryl it is actually the compelete output with full debug mode and code. — hossein hayati, May 29 '19 at 09:42
I've not. You set `ftp.set_debuglevel(2)` only after `ftp.login`. — Martin Prikryl, May 29 '19 at 10:06
Do something like `ftp = FTP()`, `ftp.set_debuglevel(2)`, `ftp.connect(host_address)`, `ftp.login(user=user_name,passwd=password)`. — Martin Prikryl, May 29 '19 at 10:25

Python: How to open files through Ftp with UTF-8/Unicode name?

0 Answers0