Fabrics put() command with hyphens

Question

I have an issue putting files to a server that contains hyphens ("-"), and I think that it may be because of how Linux is treating the file, but I am in no way sure. The script is scanning a folder for pictures/items, puts them in a list and then transferring all items to the server.

This is a part of the script:

def _transferContent(locale):
    ## Transferring images to server
    now = datetime.datetime.now()
    localImages = '/home/bcns/Pictures/upload/'
    localList = os.listdir(localImages)
    print("Found local items: ")
    print(localList)
    fname = "/tmp/backup_images_file_list"
    f = open(fname, 'r')
    remoteList = f.read()
    remoteImageLocation = "/var/www/bcns-site/pics/photos/backup_" + locale + "-" + `now.year` + `now.month` + `now.day` + "/"
    print("Server image location: " + remoteImageLocation)
    ## Checking local list against remote list (from the server)
    for localItem in localList:
        localItem_fullpath = localImages + localItem
        if os.path.exists(localItem_fullpath):
            if localItem in remoteList:
                print("Already exists: " + localItem)
            else:
                put(localItem_fullpath, remoteImageLocation)
        else:
            print("File not found: " + localItem)

And this is the out put:

Directory created successfully
/tmp/bcns_deploy/backup_images_file_list
[<server>] download: /tmp/backup_images_file_list <- /tmp/bcns_deploy/backup_images_file_list

Warning: Local file /tmp/backup_images_file_list already exists and is being overwritten.

Found local items: 
['darth-vader-mug.jpg', 'gun-dog-leash.jpg', 'think-safety-first-sign.jpg', 'hzmp.jpg', 'cy-happ-short-arms.gif', 'Hogwarts-Crest-Pumpkin.jpg']
Server image location: /var/www/bcns-site/pics/photos/backup_fujitsu-20131031/
[<server>] put: /home/bcns/Pictures/upload/darth-vader-mug.jpg -> /var/www/bcns-site/pics/photos/backup_fujitsu-20131031/

Fatal error: put() encountered an exception while uploading '/home/bcns/Pictures/upload/darth-vader-mug.jpg'

Underlying exception:
    Failure

I have tried to remove the hyphons, and then the transfer works just fine.

Server runs Ubuntu 12.04 and client runs Debian 7.1 on ext3 disks.

Irritating error, but anyone out here that has a clue on what might make this error?

How come you think the hyphen in the file name is the reason for the exception? Any more tests you did without telling us? — Ah, I see your edit ;-) — Alfe, Oct 31 '13 at 13:56
What's the remote system? Maybe it has a weird file system which does not support hyphens in file names? — Alfe, Oct 31 '13 at 14:01
Documentation of `fabric` contains an example `put`ting something to a directory called `cgi-bin/`, so I don't think hyphens are generally a problem for fabric. — Alfe, Oct 31 '13 at 14:02
The client is a Debian 7.1, and the server is an Ubuntu Server 12.04, and they are both on ext3 disks, so there shouldn't be any problems there. (post updated). — Odecif, Oct 31 '13 at 14:04
R U sure that just removing the hyphens of the basename of the file removed the problem? I'm curious because the dirname of the target also contains hyphens. Sounds just strange that removing them *after the last slash* should make a difference. — Alfe, Oct 31 '13 at 14:07
Did you try _moving_ a hyphen instead of just removing it? So, what about a file named `dar-thvadermug.jpg`? I've the feeling we're chasing ghosts here. — Alfe, Oct 31 '13 at 14:10
Found the issue! The list (backup_images_file_list) was never created, there was an old version of it lying around. When I changed the code to to update that file, it all went smooth. — Odecif, Oct 31 '13 at 14:51
Chasing ghosts, as I thought. And, sorry, but I think you did _not_ properly check whether the hyphen was the problem by removing it and introducing it again. If that file was the real reason, then you were sloppy by making sure if the hyphens were involved at all. — Alfe, Oct 31 '13 at 15:00
I believe that there is no way to prove anything else, thanks for the support anyway :-) — Odecif, Oct 31 '13 at 15:20

score 0 · Answer 1 · edited May 23 '17 at 10:32

Dashes in command line options in Linux matter, but dashes in the middle of filenames are file.

Check file permissions -- it's possible that in transferring one file manually, the perms are set differently than if Fabric transfers.

I suggest using put() to transfer a directory at a time. This will help to make sure all the files (and permissions) are what they should be.

Example (untested):

def _transferContent(locale):
    ## Transferring images to server
    now = datetime.datetime.now()
    localImageDir = '/home/bcns/Pictures/upload/'
    remoteImageDir = "/var/www/bcns-site/pics/photos/backup_" + locale + "-" + `now.year` + `now.month` + `now.day` + "/"
    print("Server image location: " + remoteImageDir)
    put( localImagesDir, remoteImageDir)

Fabrics put() command with hyphens

1 Answers1