I'm learning about python's request library so that I can automatically download some images through their links.
But the images that I'm trying to download are behind Cloudflare, and so I get ERROR 1020 Access Denied
Here's my code
import requests
from bs4 import BeautifulSoup
# -------------------------------------------------------------------------------------------------------
response = requests.get("https://main_link").text
soup = BeautifulSoup( response , 'html.parser')
for i, link in enumerate(soup.find_all('img')): # getting all image elements
l = link.get('src') # image link -> https://link/link/image.jpg
img_data = requests.get(l).content
with open(f'Test{i}.png', 'wb') as f:
f.write(img_data)
I looked at some resources like this StackOverflow question which says to use cfscrape And this is my code:
import requests
import cfscrape
from bs4 import BeautifulSoup
# ------------------------------------------------------------------------------------------------------
scraper = cfscrape.create_scraper()
response = scraper.get("https://main_link").text
soup = BeautifulSoup( response , 'html.parser')
for i, link in enumerate(soup.find_all('img')):
l = link.get('src') # https://link/link/image.jpg
img_data = scraper.get(l).content
with open(f'Test{i}.png', 'wb') as f:
f.write(img_data)
But I still get the 1020 ERROR
I even used the cloudscraper
library that too does not work.
I've looked at other resources but can't seem to understand what to do. Any help is appreciated