1

I am trying to scrape more than one page of images from bing. Currently the script below saves 28 images, but I need it to download more than this.

Here's what I'm using at the moment (poached from this article because I'm a noob):

from bs4 import BeautifulSoup
import requests
import re
import urllib2
import os

def get_soup(url):
    return BeautifulSoup(requests.get(url).text)

image_type = "ant"
query = "ant"
url = "http://www.bing.com/images/search?q=" + query + "&qft=+filterui:color2-bw+filterui:imagesize-large&FORM=R5IR3"

soup = get_soup(url)
images = [a['src'] for a in soup.find_all("img", {"src": re.compile("mm.bing.net")})]

for img in images:
    raw_img = urllib2.urlopen(img).read()
    cntr = len([i for i in os.listdir("images") if image_type in i]) + 1
    f = open("images/" + image_type + "_"+ str(cntr), 'wb')
    f.write(raw_img)
    f.close()

How can I modify the above to go through multiple pages of bing image search instead of just one page?

Harry Munro
  • 304
  • 2
  • 12
  • Possible duplicate of [Scrape multiple pages with BeautifulSoup and Python](http://stackoverflow.com/questions/26497722/scrape-multiple-pages-with-beautifulsoup-and-python) – Harrison Aug 14 '16 at 13:22
  • Multiple pages of what exactly? Ants? Something else? – Padraic Cunningham Aug 14 '16 at 21:17
  • The bind uses JavaScript to fetch more results of your search, you can investigate the requests that sends and reproduce them to your code. Or use Selenium to simulate a browser behavior that executes JS – Christos Papoulas Aug 16 '16 at 14:22
  • Yes more pictures of ants is what I was trying to achieve. Not knowing anything about Javascript is it fair to say I'm probably at a dead end with this? – Harry Munro Aug 17 '16 at 19:34

0 Answers0