How to scrape more slowly using delay python code

Question

I am making a web scraper that has a loop that goes from page to page. The only problem is that it is running too fast and I got temporarily banned from the website. (The website allows robots to access their content).

Is there any library/command that I can add to my code to make it run slower?

score 1 · Accepted Answer · answered May 14 '18 at 05:46

1

The following code will make your code sleep for 1 second.

import time
time.sleep(1)

answered May 14 '18 at 05:46

R.yan

2,214
1
16
33

Also, you can use AsyncIO. – O_o May 14 '18 at 05:53

How to scrape more slowly using delay python code

1 Answers1