Make requests using Python over Tor

Question

I want to make multiple GET requests using Tor to a webpage. I want to use a different ipaddress for each request.

import socks
import socket
socks.set_default_proxy(socks.SOCKS5, "127.0.0.1", 9150)
socket.socket = socks.socksocket
import requests
print (requests.get('http://icanhazip.com')).content

Using this, I made one request. How can I change the ipaddress to make another?

Answer 1

There are 2 aspects to your question -

1. Making requests using Tor
2. Renewing the connection as per requirement (in your case, after every request)

---

Part 1

The first one is easy to do with the latest (upwards of v2.10.0) requests library with an additional requirement of requests[socks] for using the socks proxy.

Installation -

pip install requests[socks]

Basic usage -

import requests

def get_tor_session():
    session = requests.session()
    # Tor uses the 9050 port as the default socks port
    session.proxies = {'http':  'socks5://127.0.0.1:9050',
                       'https': 'socks5://127.0.0.1:9050'}
    return session

# Make a request through the Tor connection
# IP visible through Tor
session = get_tor_session()
print(session.get("http://httpbin.org/ip").text)
# Above should print an IP different than your public IP

# Following prints your normal public IP
print(requests.get("http://httpbin.org/ip").text)

---

Part 2

To renew the Tor IP, i.e. to have a fresh visible exit IP, you need to be able to connect to the Tor service through it's ControlPort and then send a NEWNYM signal.

Normal Tor installation does not enable the ControlPort by default. You'll have to edit your torrc file and uncomment the corresponding lines.

ControlPort 9051
## If you enable the controlport, be sure to enable one of these
## authentication methods, to prevent attackers from accessing it.
HashedControlPassword 16:05834BCEDD478D1060F1D7E2CE98E9C13075E8D3061D702F63BCD674DE

Please note that the HashedControlPassword above is for the password "password". If you want to set a different password, replace the HashedControlPassword in the torrc by noting the output from tor --hash-password "<new_password>" where <new_password> is the password that you want to set.

................................................................................

Warning for Windows users: see post here.

There is an issue on windows where the setting for the controlport in the torrc file is ignored if tor was installed using the following command:

tor --service install

To resolve the issue, after editing your torrc file, type the following commands:

tor --service remove
tor --service install -options ControlPort 9051

................................................................................

Okay, so now that we have Tor configured properly, you will have to restart Tor if it is already running.

sudo service tor restart

Tor should now be up & running on the 9051 ControlPort through which we can send commands to it. I prefer to use the official stem library to control Tor.

Installation -

pip install stem

You may now renew the Tor IP by calling the following function.

Renew IP -

from stem import Signal
from stem.control import Controller

# signal TOR for a new connection 
def renew_connection():
    with Controller.from_port(port = 9051) as controller:
        controller.authenticate(password="password")
        controller.signal(Signal.NEWNYM)

To verify that Tor has a new exit IP, just rerun the code from Part 1. For some reason unknown to me, you need to create a new session object in order to use the new IP.

session = get_tor_session()
print(session.get("http://httpbin.org/ip").text)

Answer 2

Here is the code you want to use (download the stem package using pip install stem)

from stem import Signal
from stem.control import Controller

with Controller.from_port(port = 9051) as controller:
    controller.authenticate(password='your password set for tor controller port in torrc')
    print("Success!")
    controller.signal(Signal.NEWNYM)
    print("New Tor connection processed")

Good luck and hopefully that works.

Answer 3

You can try pure-python tor protocol implementation Torpy. No need original Tor client or Stem dependency at all.

$ pip3 install torpy[requests]
...

$ python3.7
>>> from torpy.http.requests import TorRequests
>>> with TorRequests() as tor_requests:
...    print("build circuit")
...    with tor_requests.get_session() as sess:
...        print(sess.get("http://httpbin.org/ip").json())
...        print(sess.get("http://httpbin.org/ip").json())
...    print("renew circuit")
...    with tor_requests.get_session() as sess:
...        print(sess.get("http://httpbin.org/ip").json())
...        print(sess.get("http://httpbin.org/ip").json())
...
build circuit
{'origin': '23.129.64.190, 23.129.64.190'}
{'origin': '23.129.64.190, 23.129.64.190'}
renew circuit
{'origin': '198.98.50.112, 198.98.50.112'}
{'origin': '198.98.50.112, 198.98.50.112'}

So each time when you getting new session you get new identity (basically you get new circuit with new exit node). See more examples at readme file https://github.com/torpyorg/torpy

Answer 4

You can use torrequest library (shameless plug). It's available on PyPI.

from torrequest import TorRequest

with TorRequest() as tr:
  response = tr.get('http://ipecho.net/plain')
  print(response.text)  # not your IP address

  tr.reset_identity()

  response = tr.get('http://ipecho.net/plain')
  print(response.text)  # another IP address, not yours

Answer 5

Requests supports proxies using the SOCKS protocol from version 2.10.0.

import requests
proxies = {
    'http': 'socks5://localhost:9050',
    'https': 'socks5://localhost:9050'
}
url = 'http://httpbin.org/ip'
print(requests.get(url, proxies=proxies).text)

Answer 6

This answer complete the one of Ashish Nitin Patil for windows (feel free to update this answer)

Part 2

ControlPort 9051
## If you enable the controlport, be sure to enable one of these
## authentication methods, to prevent attackers from accessing it.
HashedControlPassword 16:05834BCEDD478D1060F1D7E2CE98E9C13075E8D3061D702F63BCD674DE

The HashedControlPassword above is the password. If you want to set a different password in the console navigate to \Tor Browser\Browser\TorBrowser\Tor and type the following commands: tor.exe --hash-password password_XYZ | more). It will give you something like HashedControlPassword 16:54C092A8... This is your password. Now you can add it to the torrc file (Tor Browser\Browser\TorBrowser\Data\Tor\torrc).

You will need then to restart Tor:

tor --service remove
tor --service install -options ControlPort 9051

To check if that works type netstat -an you will now see that port 9051 is open.

Notice that tor --service install -... will create Tor Win32 Service. For some reason, it seems you have to stop the service to use the browser (run services.msc)

EDIT: you will find many pieces of information here (About port number & proxy, Tor, Privoxy, Auto switch user agent...).

Answer 7

This code works fine. Using Tor, it changes the IP address after each request.

import time, socks, socket
from urllib2 import urlopen
from stem import Signal
from stem.control import Controller

nbrOfIpAddresses=3

with Controller.from_port(port = 9051) as controller:
   controller.authenticate(password = 'my_pwd')
   socks.setdefaultproxy(socks.PROXY_TYPE_SOCKS5, "127.0.0.1", 9050)
   socket.socket = socks.socksocket   

   for i in range(0, nbrOfIpAddresses):
       newIP=urlopen("http://icanhazip.com").read()
       print("NewIP Address: %s" % newIP)
       controller.signal(Signal.NEWNYM)
       if controller.is_newnym_available() == False:
        print("Waitting time for Tor to change IP: "+ str(controller.get_newnym_wait()) +" seconds")
        time.sleep(controller.get_newnym_wait())
   controller.close()

Answer 8

The requests in requesocks is super old, it doesn't have response.json() and many other stuff.

I would like to keep my code clean. However, requests currently doesn't have socks5 supported yet (for more detail, read this thread https://github.com/kennethreitz/requests/pull/478)

So I used Privoxy as a http proxy that connects Tor for now.

Install and configure Privoxy on Mac

brew install privoxy
vim /usr/local/etc/privoxy/config
# put this line in the config
forward-socks5 / localhost:9050 .
privoxy /usr/local/etc/privoxy/config

Install and configure Privoxy on Ubuntu

sudo apt-get install privoxy
sudo vim /etc/privoxy/config
# put this line in the config
forward-socks5 / localhost:9050 .
sudo /etc/init.d/privoxy restart

Now I can use Tor like a http proxy. Below is my python script.

import requests

proxies = {
  'http': 'http://127.0.0.1:8118',
}

print requests.get('http://httpbin.org/ip', proxies=proxies).text

Answer 9

A good function to renew your IP. Windows example

def renew_tor_ip():
    with Controller.from_port(port = 9051) as controller:
        controller.authenticate(password="aAjkaI19!!laksjd")
        controller.signal(Signal.NEWNYM)

Example of usage

import requests
import time
from stem import Signal
from stem.control import Controller


def get_current_ip():
    session = requests.session()

    # TO Request URL with SOCKS over TOR
    session.proxies = {}
    session.proxies['http']='socks5h://localhost:9150'
    session.proxies['https']='socks5h://localhost:9150'

    try:
        r = session.get('http://httpbin.org/ip')
    except Exception as e:
        print(str(e))
    else:
        return r.text

#16:8EE7AEE3F32EEEEB605C6AA6C47B47808CA6A81FA0D76546ADC05F0F15 to aAjkaI19!!laksjd
#cmd shell "C:\Users\Arthur\Desktop\Tor Browser\Browser\TorBrowser\Tor\tor.exe" --hash-password aAjkaI19!!laksjd | more
#Torcc config
#ControlPort 9051
#HashedControlPassword 16:8EE7AEE3F32EEEEB605C6AA6C47B47808CA6A81FA0D76546ADC05F0F15

def renew_tor_ip():
    with Controller.from_port(port = 9051) as controller:
        controller.authenticate(password="aAjkaI19!!laksjd")
        controller.signal(Signal.NEWNYM)


for i in range(5):
    print(get_current_ip())
    renew_tor_ip()
    time.sleep(5)