urllib2 file name

Question

If I open a file using urllib2, like so:

remotefile = urllib2.urlopen('http://example.com/somefile.zip')

Is there an easy way to get the file name other then parsing the original URL?

EDIT: changed openfile to urlopen... not sure how that happened.

EDIT2: I ended up using:

filename = url.split('/')[-1].split('#')[0].split('?')[0]

Unless I'm mistaken, this should strip out all potential queries as well.

Do make sure you know what you want in these two cases: trailing slash (`http://example.com/somefile/`) and no path: `http://example.com` Your example will fail on the latter for sure (returning "example.com"). So will @insin's final answer. That's another reason why using urlsplit is good advice. — nealmcb, Feb 08 '12 at 23:53
from the response headers: https://stackoverflow.com/questions/11783269/python-httplib-urllib-get-filename — jozxyqk, Nov 01 '15 at 12:24
Lots of answers here miss the fact that there are two places to look for a file name: the URL and the Content-Disposition header field. All the current answers that mention the header neglect to mention that cgi.parse_header() will parse it correctly. There is a better answer here: http://stackoverflow.com/a/11783319/205212 — ʇsәɹoɈ, Oct 11 '16 at 17:09

score 49 · Accepted Answer · answered Oct 02 '08 at 15:43

49

Did you mean urllib2.urlopen?

You could potentially lift the intended filename if the server was sending a Content-Disposition header by checking remotefile.info()['Content-Disposition'], but as it is I think you'll just have to parse the url.

You could use urlparse.urlsplit, but if you have any URLs like at the second example, you'll end up having to pull the file name out yourself anyway:

>>> urlparse.urlsplit('http://example.com/somefile.zip')
('http', 'example.com', '/somefile.zip', '', '')
>>> urlparse.urlsplit('http://example.com/somedir/somefile.zip')
('http', 'example.com', '/somedir/somefile.zip', '', '')

Might as well just do this:

>>> 'http://example.com/somefile.zip'.split('/')[-1]
'somefile.zip'
>>> 'http://example.com/somedir/somefile.zip'.split('/')[-1]
'somefile.zip'

answered Oct 02 '08 at 15:43

Jonny Buchanan

61,926
17
143
150

7

Use posixpath.basename() instead of manually splitting on '/'. – Thomas Wouters Oct 02 '08 at 15:52
8

I would *always* use urlsplit() and never straight string splitting. The latter will choke if you have an URL that has a fragment or query appended, say http://example.com/filename.html?cookie=55#Section_3. – Dan Lenski Oct 02 '08 at 15:54
What about escaped characters? Should those be decoded first? – awiebe Jul 07 '17 at 07:39

score 13 · Answer 2 · answered Oct 02 '08 at 16:06

If you only want the file name itself, assuming that there's no query variables at the end like http://example.com/somedir/somefile.zip?foo=bar then you can use os.path.basename for this:

[user@host]$ python
Python 2.5.1 (r251:54869, Apr 18 2007, 22:08:04) 
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.path.basename("http://example.com/somefile.zip")
'somefile.zip'
>>> os.path.basename("http://example.com/somedir/somefile.zip")
'somefile.zip'
>>> os.path.basename("http://example.com/somedir/somefile.zip?foo=bar")
'somefile.zip?foo=bar'

Some other posters mentioned using urlparse, which will work, but you'd still need to strip the leading directory from the file name. If you use os.path.basename() then you don't have to worry about that, since it returns only the final part of the URL or file path.

Using `os.path` to parse URLs seems to rely on current OS splitting paths in the same way as URLs are split. I don't think it's guaranteed for every OS. — Rafał Dowgird, Jun 10 '13 at 21:02
This won't work on Windows. Use `import posixpath; posixpath.basename` instead. — j08lue, Jan 12 '17 at 11:49

score 7 · Answer 3 · answered Oct 02 '08 at 15:43

I think that "the file name" isn't a very well defined concept when it comes to http transfers. The server might (but is not required to) provide one as "content-disposition" header, you can try to get that with remotefile.headers['Content-Disposition']. If this fails, you probably have to parse the URI yourself.

score 6 · Answer 4 · answered Mar 20 '15 at 18:38

6

Just saw this I normally do..

filename = url.split("?")[0].split("/")[-1]

answered Mar 20 '15 at 18:38

TMF Wolfman

61
1
2

score 4 · Answer 5 · answered Mar 31 '13 at 20:05

4

Using urlsplit is the safest option:

url = 'http://example.com/somefile.zip'
urlparse.urlsplit(url).path.split('/')[-1]

answered Mar 31 '13 at 20:05

Filipe Correia

5,415
6
32
47

score 2 · Answer 6 · answered Oct 02 '08 at 15:42

Do you mean urllib2.urlopen? There is no function called openfile in the urllib2 module.

Anyway, use the urllib2.urlparse functions:

>>> from urllib2 import urlparse
>>> print urlparse.urlsplit('http://example.com/somefile.zip')
('http', 'example.com', '/somefile.zip', '', '')

Voila.

Régis B. · Answer 7 · 2016-06-26T20:52:07.913

2

The os.path.basename function works not only for file paths, but also for urls, so you don't have to manually parse the URL yourself. Also, it's important to note that you should use result.url instead of the original url in order to follow redirect responses:

import os
import urllib2
result = urllib2.urlopen(url)
real_url = urllib2.urlparse.urlparse(result.url)
filename = os.path.basename(real_url.path)

edited Jun 26 '16 at 20:52

answered May 11 '15 at 06:15

Régis B.

10,092
6
54
90

Yth · Answer 8 · 2016-04-28T14:52:52.013

2

You could also combine both of the two best-rated answers : Using urllib2.urlparse.urlsplit() to get the path part of the URL, and then os.path.basename for the actual file name.

Full code would be :

>>> remotefile=urllib2.urlopen(url)
>>> try:
>>>   filename=remotefile.info()['Content-Disposition']
>>> except KeyError:
>>>   filename=os.path.basename(urllib2.urlparse.urlsplit(url).path)

edited Apr 28 '16 at 14:52

answered Apr 28 '16 at 14:41

Yth

146
1
5

score 1 · Answer 9 · answered Oct 02 '08 at 15:45

I guess it depends what you mean by parsing. There is no way to get the filename without parsing the URL, i.e. the remote server doesn't give you a filename. However, you don't have to do much yourself, there's the urlparse module:

In [9]: urlparse.urlparse('http://example.com/somefile.zip')
Out[9]: ('http', 'example.com', '/somefile.zip', '', '', '')

score 1 · Answer 10 · answered Oct 02 '08 at 15:46

1

not that I know of.

but you can parse it easy enough like this:

url = 'http://example.com/somefile.zip'
print url.split('/')[-1]

answered Oct 02 '08 at 15:46

Corey Goldberg

59,062
28
129
143

score 0 · Answer 11 · edited Apr 29 '16 at 16:45

0

import os,urllib2
resp = urllib2.urlopen('http://www.example.com/index.html')
my_url = resp.geturl()

os.path.split(my_url)[1]

# 'index.html'

This is not openfile, but maybe still helps :)

edited Apr 29 '16 at 16:45

Blexy

9,573
6
42
55

answered Oct 02 '08 at 15:45

user15453

466
1
5
14

score 0 · Answer 12 · edited Jul 11 '14 at 12:45

using requests, but you can do it easy with urllib(2)

import requests
from urllib import unquote
from urlparse import urlparse

sample = requests.get(url)

if sample.status_code == 200:
    #has_key not work here, and this help avoid problem with names

    if filename == False:

        if 'content-disposition' in sample.headers.keys():
            filename = sample.headers['content-disposition'].split('filename=')[-1].replace('"','').replace(';','')

        else:

            filename = urlparse(sample.url).query.split('/')[-1].split('=')[-1].split('&')[-1]

            if not filename:

                if url.split('/')[-1] != '':
                    filename = sample.url.split('/')[-1].split('=')[-1].split('&')[-1]
                    filename = unquote(filename)

score 0 · Answer 13 · answered Sep 10 '15 at 22:31

You probably can use simple regular expression here. Something like:

In [26]: import re
In [27]: pat = re.compile('.+[\/\?#=]([\w-]+\.[\w-]+(?:\.[\w-]+)?$)')
In [28]: test_set 

['http://www.google.com/a341.tar.gz',
 'http://www.google.com/a341.gz',
 'http://www.google.com/asdasd/aadssd.gz',
 'http://www.google.com/asdasd?aadssd.gz',
 'http://www.google.com/asdasd#blah.gz',
 'http://www.google.com/asdasd?filename=xxxbl.gz']

In [30]: for url in test_set:
   ....:     match = pat.match(url)
   ....:     if match and match.groups():
   ....:         print(match.groups()[0])
   ....:         

a341.tar.gz
a341.gz
aadssd.gz
aadssd.gz
blah.gz
xxxbl.gz

score 0 · Answer 14 · answered Apr 11 '16 at 19:28

Using PurePosixPath which is not operating system—dependent and handles urls gracefully is the pythonic solution:

>>> from pathlib import PurePosixPath
>>> path = PurePosixPath('http://example.com/somefile.zip')
>>> path.name
'somefile.zip'
>>> path = PurePosixPath('http://example.com/nested/somefile.zip')
>>> path.name
'somefile.zip'

Notice how there is no network traffic here or anything (i.e. those urls don't go anywhere) - just using standard parsing rules.

urllib2 file name

14 Answers14

Linked

Related