0

I'm coding a small project for web scraping and the program scraps a shop that contains multiple pages. It's a little bit hard to get some things to be clicked, so for me it's a lot easier to get their URL with Beautifulsoup and just to do driver.get(URL). I've been recently kicked out from website because of the multiple urlopen with requests (earlier), that's why I started to use selenium. My question is: Would be driver.get(URL) seen as a most likely "human-action" like clicking on a button? Would be driver.get as risky as urlopen, so that my driver will be kicked out?

I hope you understood my question. Thanks in advance.

ressy
  • 7
  • 1
  • 5

1 Answers1

0

I think if you use time.sleep to wait between each action it is not going to detect you as a bot

import time

driver.get(url)
#wait 5 secconds
time.sleep(5)
#do an other action
rafalou38
  • 576
  • 1
  • 5
  • 16