0

I'm trying to learn web scraping. so what i'm trying to do is insert a category link myself ex: https://www.amazon.in/TVs/b/ref=nav_shopall_sbc_tvelec_television?ie=UTF8&node=1389396031

now go to division having class 's-item-container' now read the url inside tag having class 'a-link-normal s-access-detail-page s-color-twister-title-link a-text-normal' open the url get the relevant details come back and repeat for other divisions having s-item-container as class

But i don't know how to do it. the video i'm learning from is showing me how to extract from the list page only. if someone can show me the code i can study that.

Utkarsh Tomar
  • 1
  • You have to use css selectors to scrape, you should probably do some reading about how to select elements by css selector. For a quick example, in order to select the div with class s-item-container. You can use ".s-item-container" this will select every element in the dom with class s-item-container. – sjdm Oct 03 '18 at 23:50
  • Here is a reference: https://www.w3schools.com/cssref/css_selectors.asp – sjdm Oct 03 '18 at 23:50
  • i can do that but what i cant do is open the product details page and come back to list page after getting the details. i can select divisions with classes and ids – Utkarsh Tomar Oct 03 '18 at 23:52
  • @UtkarshTomar you have to use callback, check this answer https://stackoverflow.com/questions/22648475/understanding-callbacks-in-scrapy – Sohan Das Oct 04 '18 at 04:23

0 Answers0