0

Hi I am using selenium with python by making auto google image download by

using 'with ...as' to download image because some website block the crawling request, so I have to add cookie and header when download it can't download image with python

but the problem it is only downloaded 0 byte images when I use 'for image in images' inside

I thought that it was the problem getting url part but it was not

    count = 1
    images = driver.find_elements_by_css_selector(".rg_i.Q4LuWd")
for image in images: 
    try:
        image.click()
        time.sleep(2)        
        imgUrl=driver.find_element_by_xpath('/html/body/div[2]/c-wiz/div[3]/div[2]/div[3]/div/div/div[3]/div[2]/c-wiz/div/div[1]/div[1]/div/div[2]/a/img').get_attribute("src") 
        cookies = dict(BCPermissionLevel='PERSONAL')
        count+=1
        with open(str(count)+'.jpg', 'wb') as handle:
            response = requests.get(pic_url, headers={"User-Agent": "Mozilla/5.0"}, cookies=cookies,stream=True)
            if not response.ok:
                    print (response)
            for block in response.iter_content(1024):
                    if not block:
                            break
                    handle.write(block)
        time.sleep(2)
    except:
        pass

If you need full code to answer here please. https://github.com/yeon333/crawl-download-ing/blob/master/combate.py

yeeoon
  • 13
  • 5
  • There are a few possibilities: 1. wait is not long enough (see https://selenium-python.readthedocs.io/waits.html) 2. `find_element_by_xpath` is fragile, try something more robust like by id or by class – Tim Feb 17 '21 at 03:44
  • images=driver.find_elements_by_css_selector(".rg_i.Q4LuWd") might also be a dynamic class name so check if it's printing the right number of images found. – Arundeep Chohan Feb 17 '21 at 04:28

0 Answers0