Create missing directories in ftplib storbinary

Question

I was using pycurl to transfer files over ftp in python. I could create the missing directories automatically on my remote server using:

c.setopt(pycurl.FTP_CREATE_MISSING_DIRS, 1)

for some reasons, I have to switch to ftplib. But I don't know how to to the same here. Is there any option to add to storbinary function to do that? or I have to create the directories manually?

score 22 · Accepted Answer · answered May 22 '12 at 05:03

FTP_CREATE_MISSING_DIRS is a curl operation (added here). I'd hazard a guess that you have to do it manually with ftplib, but I'd love to be proven wrong, anyone?

I'd do something like the following: (untested, and need to catch ftplib.all_errors)

ftp = ... # Create connection

# Change directories - create if it doesn't exist
def chdir(dir): 
    if directory_exists(dir) is False: # (or negate, whatever you prefer for readability)
        ftp.mkd(dir)
    ftp.cwd(dir)

# Check if directory exists (in current location)
def directory_exists(dir):
    filelist = []
    ftp.retrlines('LIST',filelist.append)
    for f in filelist:
        if f.split()[-1] == dir and f.upper().startswith('D'):
            return True
    return False

Or you could do directory_exists like this: (a bit harder to read?)

# Check if directory exists (in current location)
def directory_exists(dir):
    filelist = []
    ftp.retrlines('LIST',filelist.append)
    return any(f.split()[-1] == dir and f.upper().startswith('D') for f in filelist)

Thank you, although it was not exactly what I was looking for, but it was a good answer. Thanx ;) — AliBZ, May 29 '12 at 18:12
No, you don't have to do it manually. You could just call the `makedirs` method in the `ftputil` package instead. — xApple, Jul 09 '14 at 17:49
Note this solution only works with top level directories and does not work with sub-directories — Mykhailo Seniutovych, Mar 15 '19 at 10:39

lecnt · Answer 2 · 2014-09-16T13:40:33.430

20

I know it's kind of an old post but I just needed this and came up with a very simple function. I'm new to Python so I'd appreciate any feedback.

from ftplib import FTP

ftp = FTP('domain.com', 'username', 'password')

def cdTree(currentDir):
    if currentDir != "":
        try:
            ftp.cwd(currentDir)
        except IOError:
            cdTree("/".join(currentDir.split("/")[:-1]))
            ftp.mkd(currentDir)
            ftp.cwd(currentDir)

Usage example:

cdTree("/this/is/an/example")

edited Sep 16 '14 at 13:40

answered Aug 20 '13 at 18:14

lecnt

353
1
3
9

3

very nice ! `dir` is a python built-in you might want to change that variable name... also you want to catch specific exceptions, not all of them – xApple Jul 09 '14 at 16:20
Thanks you xApple for your feedback. I replaced 'dir' and restricted to only catch IOError exceptions. – lecnt Aug 12 '14 at 17:43
I think you forgot to replace an instance of the `dir` variable. – xApple Aug 13 '14 at 21:51
Oops, fixed it. Thank you for catching it. Now it's perfect, it just needs to be up voted. :) – lecnt Sep 16 '14 at 13:36
@lecnt - a couple more suggestions: `lower_case_with_underscores` is preferred for function/variable names - see [pep8](https://www.python.org/dev/peps/pep-0008/#function-names). Also it's better to use `os.path` functions for manipulating paths, e.g. `os.path.normpath(os.path.join(current_dir, '..'))` – Alex L Mar 14 '17 at 13:13
smart solution. Exactly what i was looking for.! – sushh Oct 31 '18 at 04:11
I'd like to highlight that this also answers another common question: how do I check if a ftp directory exists. I think this way is more Pythonic than the answer of listing the directory contents then looking for the directory name. Instead, try to change to that directory, and catch the error, in line with "ask for forgiveness" rather than "ask for permission". – Julian Drago Dec 05 '19 at 21:58
In my opinion `try` and `except` is very pythonic. This question deals with it: https://stackoverflow.com/questions/7604636 – tommy.carstensen Apr 18 '20 at 03:32
2

This no longer works (with me). I had to change `ioerror` to `ftplib.error_perm`, and `ftp.mkd(currentDir)` to `ftp.mkd(currentDir.split("/")[-1])`. – dloeckx Apr 19 '20 at 13:16
even in 2022 this is the best way of doing it – Nelson Sequiera Jul 23 '22 at 21:35

score 4 · Answer 3 · answered Mar 22 '13 at 01:52

I tried adding this as a comment to the @Alex L 's answer, but it was too long. You need to descend recursively when changing directory if you want to create directories on the way. E.g.

def chdir(ftp, directory):
    ch_dir_rec(ftp,directory.split('/')) 

# Check if directory exists (in current location)
def directory_exists(ftp, directory):
    filelist = []
    ftp.retrlines('LIST',filelist.append)
    for f in filelist:
        if f.split()[-1] == directory and f.upper().startswith('D'):
            return True
    return False

def ch_dir_rec(ftp, descending_path_split):
    if len(descending_path_split) == 0:
        return

    next_level_directory = descending_path_split.pop(0)

    if not directory_exists(ftp,next_level_directory):
        ftp.mkd(next_level_directory)
    ftp.cwd(next_level_directory)
    ch_dir_rec(ftp,descending_path_split)

score 2 · Answer 4 · answered Sep 10 '20 at 13:57

I am using the following lines to resolve missing directory paths for FTP file copy

import os
ftps = FTP_TLS('ftps_server')
ftps.connect()
ftps.login()

destination_dir_path = 'some/dir/path'          # directory path on FTP
dir_path = ''
for i in destination_dir_path.split('/'):
    dir_path = os.path.join(dir_path,i)
    if i not in ftps.nlst(os.path.dirname(dir_path)):
        ftps.mkd(dir_path)                      # create directory on the FTP
ftps.storbinary(...)                            # store file using the binary mode

score 1 · Answer 5 · answered Mar 23 '21 at 10:43

An alternative is to simply loop through each of the path elements, create the next and change into the newly-created directory. My use case was fairly straightforward though as I was copying items from one FTP server to another.

def create_ftp_path(session: ftplib.FTP, required_dir: str):
    required_dir = required_dir.split('/')[:-1]
    for path_item in required_dir:
        if path_item.strip() == '':
            continue
        path_item = path_item.replace('/', '')
        try:
            session.cwd(path_item)
        except:
            session.mkd(path_item)
            session.cwd(path_item)

Considerations:

This function assumes you have already changed directory for your FTP session to some base path and the required_dir is a path from that base path.
required_dir includes file name as the last element.
I'm removing any / characters because in my case they were causing 553 permission denied exception.
The exception handling is lacking, but in my case upload validation is happening further in the code so even if it fails it will be caught further down.

score 1 · Answer 6 · 2022-03-24T19:16:48.903

A more robust and reliable solution:

using ftplib

Hostname = yourhostname.com
Username = yourusername
Password = yourpassword

def mkdirs(path):
    ftp = FTP(Hostname,Username,Password) 
    items = path.split('/')
    cwd = "/"
    for i in range(len(items)):
        list = ftp.nlst()
        if(not '.' in items[i] and not items[i] in list):
            ftp.mkd(cwd + items[i] + "/")
            cwd += items[i] + '/'
            ftp.cwd(cwd)
    ftp.quit()

mkdirs('path/to/directory/file.name')

This will create directories on your server if they do not exist.

Limitations: This will not work on folders with names that contain ..

score 0 · Answer 7 · answered Mar 20 '14 at 10:35

This code will create all missing folders in path:

...

def chdir(ftp_path, ftp_conn):
    dirs = [d for d in ftp_path.split('/') if d != '']
    for p in dirs:
        print(p)
        check_dir(p, ftp_conn)


def check_dir(dir, ftp_conn):
    filelist = []
    ftp_conn.retrlines('LIST', filelist.append)
    found = False

    for f in filelist:
        if f.split()[-1] == dir and f.lower().startswith('d'):
            found = True

    if not found:
        ftp_conn.mkd(dir)
    ftp_conn.cwd(dir)

if __name__ == '__main__':
    ftp_conn = ... # ftp connection
    t = 'FTP/for_Vadim/1/2/3/'

    chdir(t, ftp_conn)

This code will check all dirs in path and create missing dirs

before "FTP/for_Vadim/" after "FTP/for_Vadim/1/2/3/"

score 0 · Answer 8 · answered Sep 07 '18 at 12:41

I'm using something like this (without cwd):

# -*- coding:utf-8 -*-

from ftplib import FTP, error_perm


def createDirs(ftp, dirpath):
    """
    Create dir with subdirs.

    :param ftp:     connected FTP
    :param dirpath: path (like 'test/test1/test2')

    :type ftp:      FTP
    :type dirpath:  str
    :rtype:         None

    """

    dirpath = dirpath.replace('\\', '/')
    tmp = dirpath.split('/')
    dirs = []

    for _ in tmp:
        if len(dirs) == 0:
            dirs.append(_)
            continue

        dirs.append(dirs[-1] + '/' + _)

    for _ in dirs:
        try:
            ftp.mkd(_)
        except error_perm as e:
            e_str = str(e)
            if '550' in e_str and 'File exists' in e_str:
                continue


if __name__ == '__main__':
    # init ftp
    createDirs(ftp=ftp, dirpath='test/1/2/3')

Create missing directories in ftplib storbinary

8 Answers8

Linked