0

I want to scrape rusprofile.ru using Selenium chromedriver Python. This site has a recaptcha, I know that. I want to connect a captcha recognition service.

But for some reason, captcha appears only in my personal browser. At this time, the browsers that I enter using Selenium Python get HTTP ERROR 429.

And only when I confirm the captcha in a personal browser, I can go to the site using Selenium. How can I make captcha appear in Selenium chromedriver browser?

url = 'https://www.rusprofile.ru/search?query=1027806061425&search_inactive=0'
options = webdriver.ChromeOptions()
options.add_argument('log-level=3')
options.add_argument('headless')
options.add_argument('User-Agent=Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML like Gecko) Chrome/35.0.1916.153 Safari/537.36')
options.add_argument('user-data-dir=selenium')
driver = webdriver.Chrome(chrome_options=options,)
driver.get(url)

try:
    capcha_text = driver.find_element_by_xpath('//p[@text()="Активность с вашего IP-адреса была распознана как автоматическая. Нам необходимо убедиться, что вы человек."]')
    print('CAPCHA IS HERE')
    captcha = driver.find_element_by_xpath('//p[@class="g-recaptcha"]')
    
    print('CAPCHA IS HERE X2')
    
except Exception as e:
    print(str(e))
DisappointedByUnaccountableMod
  • 6,656
  • 4
  • 18
  • 22
FleXX
  • 129
  • 1
  • 7
  • 1
    [HTTP 429](https://en.wikipedia.org/wiki/List_of_HTTP_status_codes#4xx_Client_errors) is "too many requests". You're probably hammering the server. – ChrisGPT was on strike Feb 15 '20 at 14:24
  • What should I do if this error appears? This error disappears after I enter the site through my browser (not through Selenium) and confirm the captcha. But I want to automate the process, for example, by connecting services to recognize captcha. – FleXX Feb 15 '20 at 14:30
  • 1
    Slow down. You're being rate limited. Also, CAPTCHAs are there to prevent _exactly_ what you're trying to do. They're not trivial to circumvent, and if you manage to do it technically you're probably breaking the site's user agreement. I advise you not to do that. – ChrisGPT was on strike Feb 15 '20 at 14:35
  • OK, I'll set rate limits. But now the captcha does not need to be confirmed :) Thank you for advise – FleXX Feb 15 '20 at 14:41
  • FYI, if you want to use the default profile (your chrome profile) as part of selenium you can follow [this post](https://stackoverflow.com/questions/56344560/selenium-point-towards-default-chrome-session/56402113#56402113) – supputuri Feb 15 '20 at 19:34

0 Answers0