0

I have to concatenate a hardcoded path of "string" type to a URL to have a result which is a URL.

url (which doesn't end with "/") + "/path/to/file/" = new_url

I tried concatenation using URL join and also tried used simple string concat but the result is not a URL which can be reached. (not that the URL address is invalid )

mirror_url = "http://amazonlinux.us-east- 
2.amazonaws.com/2/core/latest/x86_64/mirror.list"

response = requests.get(mirror_url)
contents_in_url = response.content


## returns a URL as shown below but of string type which cannot be 
##concatenated to another string type which could be requested as a valid 
##URL.
'http://amazonlinux.us-east- 2.amazonaws.com/2/core/2.0/x86_64/8cf736cd3252ada92b21e91b8c2a324d05b12ad6ca293a14a6ab7a82326aec43'

path_to_add_to_url = "/repodata/primary.sqlite.gz"

final_url = contents_in_url + path_to_add_to_url

Desired Result:

Without omitting any path to that file.

final_url = "http://amazonlinux.us-west-2.amazonaws.com/2/core/2.0/x86_64/8cf736cd3252ada92b21e91b8c2a324d05b12ad6ca293a14a6ab7a82326aec43/repodata/primary.sqlite.gz"
prash
  • 13
  • 1
  • 4
  • So what is the result you are getting? Have you looked at https://stackoverflow.com/questions/1793261/how-to-join-components-of-a-path-when-you-are-constructing-a-url-in-python/15279799 ? – intotecho Jul 19 '19 at 04:51
  • @intotecho Yeah,but in my case I have to stick to "requests" library ,cant use 'urllib'. – prash Jul 19 '19 at 12:39

1 Answers1

0

You need to get contents of the first response by response.text method, not response.content:

import requests

mirror_url = "http://amazonlinux.us-east-2.amazonaws.com/2/core/latest/x86_64/mirror.list"

response = requests.get(mirror_url)
contents_in_url = response.text.strip()

path_to_add_to_url = "/repodata/primary.sqlite.gz"

response = requests.get(contents_in_url + path_to_add_to_url)

with open('primary.sqlite.gz', 'wb') as f_out:
    f_out.write(response.content)

print('Downloading done.')
Andrej Kesely
  • 168,389
  • 15
  • 48
  • 91
  • the 'mirror_url' in the question when requested gives a string which when concatenated with '/repodata/primary.sqlite.gz' returns a string ,which can't be requested to get response 200. – prash Jul 19 '19 at 12:47
  • Kesely Thanks, but unfortunately that doesn't seem to work in my case as the 'mirror_url' also seems to have a URL in its contents. Strangely so it doesn't work. – prash Jul 19 '19 at 14:37
  • The below code doesn't seem to work. ` import requests url = "http://amazonlinux.us-east- 2.amazonaws.com/2/core/latest/x86_64/mirror.list" response = requests.get(url) text = response.content repo_path = "/repodata/primary.sqlite.gz" res = requests.get(text + repo_path) print res ` – prash Jul 19 '19 at 16:55
  • @prash Updated my answer – Andrej Kesely Jul 19 '19 at 17:01