0

I scraping some pages and these pages check my IP if it is a vpn or proxy (fake IP) if it is found fake the site is blocking my request please if there is a way to change my IP every x time with real IP Without using vpn or proxy or restart router Note: I am using a Python script for this process

John Walker
  • 1
  • 1
  • It is unlikely to change your IP address without intermediate servers such as VPN or proxies. You can consider residential proxies with rotating API for your Python script. – Michael C. Nov 17 '21 at 00:22

2 Answers2

1

You IPAddress is fixed by your internet service provider, if you reset your home router, u sometimes can take another IPAddress depending on various internal questions. Some Websites, block by the User-Agent, IP GeoLocation of your request or by rate limit.. but if u sure its is by IP, so the only way to swap your IPAddress is through by VPNTunneling or ProxyMesh.

Murilo Chianfa
  • 36
  • 3
1

You can obtain free proxy address from https://www.freeproxylists.net/ . Since these are free proxies so it may get down quickly so sometime you might need to rotate ip with each request you made to your target address.

You can set proxy address, Please follow up this question, how to set proxy, Proxies with Python 'Requests' module

So the flow would be:

  1. Scrape the proxies from above address first.
  2. Then add the proxy header as mentioned in the another question.
  3. Rotate Ip with another request to target.

There are certain blocking factor not only your ip.

  1. Like browser agent (https://www.scrapehero.com/how-to-fake-and-rotate-user-agents-using-python-3/?sfw=pass1637120088).
  2. Too rigorous scraping (try to randomize timing of scraping between two requests).
  3. Not following up robots.txt file (this sometime cant be avoided).
DisappointedByUnaccountableMod
  • 6,656
  • 4
  • 18
  • 22
Gaurav
  • 533
  • 5
  • 20