List all the files in all subdirectories from an FTP using Python

Question

I'm new to Python and I'm trying to list all the files in all the sub-directories from an FTP. The FTP, as usual, is in this format.

 A
 B
 C

Subdirectories :

 AA
 BB
 CC

I could list the directories ['A', 'B', 'C'] using ftp.nlist(). I'd like to get ['AA', 'BB', 'CC'] as my output. I've tried and looked up a lot to find a solution/hint to do this.

You could use FTP.retrlines() - https://docs.python.org/2/library/ftplib.html#ftplib.FTP.retrlines — shaktimaan, Aug 17 '14 at 18:48
@shaktimaan Thanks. But that does something very similar to ftp.dir. It just lists the directories. I want the list of sub-directories. — abn, Aug 17 '14 at 19:25

score 4 · Answer 1 · answered Jul 20 '15 at 08:47

I know this is a bit old, but an answer here could have saved me a bit of effort, so here it is. I'm a bit of an amateur, so this is probably not the most efficient way, but here is a program I wrote to get all directories on an FTP server. It will list all directories no matter how far they are down the tree.

from ftplib import FTP

def get_dirs_ftp(folder=""):
    contents = ftp.nlst(folder)
    folders = []
    for item in contents:
        if "." not in item:
            folders.append(item)
    return folders

def get_all_dirs_ftp(folder=""):
    dirs = []
    new_dirs = []

    new_dirs = get_dirs_ftp(folder)

    while len(new_dirs) > 0:
        for dir in new_dirs:
            dirs.append(dir)

        old_dirs = new_dirs[:]
        new_dirs = []
        for dir in old_dirs:
            for new_dir in get_dirs_ftp(dir):
                new_dirs.append(new_dir)

    dirs.sort()
    return dirs


host ="your host"
user = "user"
password = "password"

print("Connecting to {}".format(host))
ftp = FTP(host)
ftp.login(user, password)
print("Connected to {}".format(host))

print("Getting directory listing from {}".format(host))
all_dirs = get_all_dirs_ftp()
print("***PRINTING ALL DIRECTORIES***")
for dir in all_dirs:
    print(dir)

The code above is quite slow, is there a better solution to get all the sub dirs? — wawawa, Aug 09 '21 at 13:07

score 1 · Answer 2 · answered Jul 06 '21 at 13:33

I wrote a similar solution to Ed Kern, but using the "mlsd" command. Since Ed Kern`s code would cause an error for files without a filename extension. Using mlsd this error is avoided. Note that very old FTP servers might not have the mlsd command.

from ftplib import FTP

def get_items_mlsd(folder):
    filedatas = []
    for file_data in ftp.mlsd(folder):
        filedatas.append(file_data)
    return filedatas

def get_all_dirs_ftp(folder=""):
    items = []
    new_items = []

    new_items = get_items_mlsd(folder)

    while len(new_items) > 0:
        old_dirs = new_items
        new_items = []
        for file_data in old_dirs:
            file_name, meta = file_data
            file_type = meta.get("type")
            if file_type != "dir":
                items.append(file_name)
            else:
                news = get_items_mlsd(file_name)
                for new in news:
                    file_name1 , meta = new
                    file_type = meta.get("type")
                    if file_type == "dir":
                        new = list(new)
                        directory = new[0]
                        new[0] = file_name + "/" + directory
                        new = tuple(new)
                        new_items.append(new)
                    else:
                        file_name1 = file_name + "/" + file_name1
                        items.append(file_name1)
    items.sort()
    return items

host = "host"
user = "user_name"
password = "pw"

print("Connecting to {}".format(host))
ftp = FTP(host)
ftp.login(user, password)
print("Connected to {}".format(host))
print("Getting file listing from {}".format(host))

all_items = get_all_dirs_ftp()

print("***PRINTING ALL ITEMS***")
with open('ftp_files.txt', 'w') as f:
    for dir in all_items:
        print(dir)

List all the files in all subdirectories from an FTP using Python

2 Answers2