0

I am trying to work out why my Python open call says a file doesn't exist when it does. If I enter the exact same file url in a browser the photo appears.

The error message I get is:

No such file or directory: 'https://yhistory.s3.amazonaws.com/media/userphotos/1_1471378042183_cdv_photo_033.jpg'

The Python code is:

full_path_filename = 'https://' + settings.AWS_STORAGE_BUCKET_NAME + '.s3.amazonaws.com/' + file_name
fd_img = open(full_path_filename, 'r')

I was wondering if this problem is something to do with the file being in an AWS S3 bucket but I am able to connect to the bucket and list its contents.

Andy
  • 49,085
  • 60
  • 166
  • 233
Bill Noble
  • 6,466
  • 19
  • 74
  • 133
  • Is that the name of the file on your machine, or are you trying to access it over the internet? – Patrick Haugh Nov 29 '16 at 20:21
  • If you're trying to access files over the internet you need to use a module like `urllib2` or `requests` – Patrick Haugh Nov 29 '16 at 20:23
  • The Python code is running on my laptop connected to the internet. – Bill Noble Nov 29 '16 at 20:24
  • 4
    The `open` function is for opening files that are reachable via pathnames. `https://yhistory.s3.amazonaws.com/media/userphotos/1_1471378042183_cdv_photo_033.jpg` is a URL, not a pathname, so `open` doesn't work for it. You have to use a function from a module explicitly for requests over the internet – Patrick Haugh Nov 29 '16 at 20:27
  • http://stackoverflow.com/questions/7391945/how-do-i-read-image-data-from-a-url-in-python – r0xette Nov 29 '16 at 20:36
  • Is requests.get(url) a direct replacement for open(url)? I am doing the open to do some image processing using pillow – Bill Noble Nov 29 '16 at 20:49
  • @Bill, no it is not a direct replacement. `requests.get(url)` returns a response object which contains the file. Then you need to extract the file to your local filesystem, modify it, and re-upload using `requests`. Check my comment under theClonerx's answer for more clues. – Juan Tomas Nov 29 '16 at 21:05

1 Answers1

3

If you are trying to open a file over internet you should do something like this (assuming that you are using python 3):

import urllib.request
full_path_filename = 'https://' + settings.AWS_STORAGE_BUCKET_NAME + '.s3.amazonaws.com/' + file_name

file = urllib.request.urlopen(full_path_filename)

This will download the actual file. You can use file as an another file like object.

TheClonerx
  • 343
  • 2
  • 13
  • 2
    Python `requests` is a powerful HTTP library that conveniently handles difficult stuff like cookies and SSL. Refer to [this question](http://stackoverflow.com/questions/13137817/how-to-download-image-using-requests) for some examples of how to deconstruct `requests` response objects. Also, since you're using a service where you probably have to be logged in, you may need to make two transactions: one to provide your login credentials and get the session cookie, and another to fetch file(s). – Juan Tomas Nov 29 '16 at 20:32