To get redirected URL with requests

Question

I want to get redırected URL with requests. My URL is https://twitter.com/i/user/2274951674 . When I enter this url my browser, The URL redirect https://twitter.com/ozanbayram01

r=requests.get("https://twitter.com/i/user/2274951674")`

r.url #doesnt work

did you try reading the documentation? I believe they mention r.history — Chris, Jun 21 '20 at 20:15
`r.url` should be the url that was _requested_. To look at redirects in the response, use `r.history`. — John Gordon, Jun 21 '20 at 20:29
It's not redirecting. I suspect it's just using javascript or something else to modify the url in your browser. Perhaps use selenium and capture the url in the browser after the page finishes loading. — Chris, Jun 21 '20 at 22:53
Is there any way without using selenium? I can use requests-html instead of requests. — Ozan Bayram, Jun 22 '20 at 03:56

score 1 · Answer 1 · answered Jan 26 '22 at 19:29

I didn't have any luck to get redirected url back by using requests as mentioned in previous posts. But I was able to work around this using webbrowser library and then get the browser history using sqlite 3 and was able to get the result that you are looking for.

let me know if you found another solutions.

here is my code:

import webbrowser
import sqlite3
import pandas as pd
import shutil

webbrowser.open("https://twitter.com/i/user/2274951674")
#source file is where the history of your webbroser is saved, I was using chrome, but it should be the same process if you are using different browser
source_file = 'C:\\Users\\{your_user_id}\\AppData\\Local\\Google\\Chrome\\User Data\\Default\\History'
# could not directly connect to history file as it was locked and had to make a copy of it in different location
destination_file = 'C:\\Users\\{user}\\Downloads\\History'
time.sleep(30) # there is some delay to update the history file, so 30 sec wait give it enough time to make sure your last url get logged
shutil.copy(source_file,destination_file) # copying the file.
con = sqlite3.connect('C:\\Users\\{user}\\Downloads\\History')#connecting to browser history
cursor = con.execute("SELECT * FROM urls")
names = [description[0] for description in cursor.description]
urls = cursor.fetchall()
con.close()
df_history = pd.DataFrame(urls,columns=names)
last_url = df_history.loc[len(df_history)-1,'url']
print(last_url)

>>https://twitter.com/ozanbayram01

score 0 · Answer 2 · answered Jul 07 '23 at 17:12

Here's the relevant part in the Requests documentation.

Use the history attribute of the Response object. It's a list of Response objects starting from oldest to most recent. So if from-here.com redirects to to-there.com, then it looks like this:

>>> import requests
>>> res = requests.get('https://from-here.com')
>>> res.url
'https://from-here.com'
>>> res.history[0]
<Response [302]>
>>> res.history[0].url
'https://to-there.com'

To get redirected URL with requests

2 Answers2

Linked