1

I am trying to scrape an ecommerce site and search for its products. I have done the same thing for two other sites, it worked there. But for this one I'm getting this error.

def parse_url(self,url):
    hdr = {'User-Agent':'Mozilla/5.0'}
    req = urllib2.Request(url,headers=hdr)
    res = urllib2.urlopen(req)

    data=src.read()
    soup=BeautifulSoup(data,"lxml")
    return soup
 def scrape(self,name):
    k=name
    url ="http://www.snapdeal.com/search?keyword=" + str(k) + "&santizedKeyword=&catId=&categoryId=175&suggested=true&vertical=p&noOfResults=20&clickSrc=suggested&lastKeyword=&prodCatId=&changeBackToAll=false&foundInAll=false&categoryIdSearched=&cityPageUrl=&url=&utmContent=&catalogID=&dealDetail="

    soup=self.parse_url(url)
sneha
  • 63
  • 1
  • 3
  • It's probably about that website, if you tried this before and worked, then your codes are fine. – GLHF Dec 27 '14 at 11:47
  • It was because the string had to be escaped before concatenating with the URL. I found the answer here. http://stackoverflow.com/questions/8840303/urllib2-http-error-400-bad-request?rq=1 – sneha Dec 27 '14 at 12:07

0 Answers0