0

I'm new to scrapy/python, could you please help me? I'm trying to get the name of each workshop in this website using scrapy, but still no result, so I cant't go beyond the tag. Here's my code:

import scrapy
class OffSpider(scrapy.Spider):
    name = "offs"
    start_urls = ['https://oficinasonline.com.br/oficinas/busca?state_id=26']

    def parse(self, response):
        for off in response.css('.workshop-list ul'):
            yield {
                'name': off.css('h4 ::text'),
            }

Thanks.

Bruno Teraoka
  • 1
  • Possible duplicate of [Scraping dynamic content using python-Scrapy](https://stackoverflow.com/questions/30345623/scraping-dynamic-content-using-python-scrapy) – Jeff Mercado Jul 24 '18 at 21:15

1 Answers1

0

Checking the part you select with .workshop-list: 

                        <div class="workshop-list">
                            <ul class="block-grid-sm-2 block-grid-xs-1"></ul>
                            <div class="get-more-box" style="display:none;">
                                <a href="#" class="go-button-blue get-more">
                                    <div class="spinner-box"><div class="spinner "></div></div>                                        <span class="text" data-start="Carregar mais" data-end="Não há mais oficinas">Carregar mais</span>
                                </a>
                            </div>
                        </div>

With your selector you get only this part: 

<ul class="block-grid-sm-2 block-grid-xs-1"></ul>

So your selector for the for loop is to selective.

If you want to get "Carregar mais" you can select the value in the span tag:

This is: 

response.css('.workshop-list')
....
    'name': off.css('span::text'),

If you want to get "block-grid-sm-2 block-grid-xs-1" the selector can be: 

response.css('.workshop-list ul')
....
    'classname': off.xpath('@class')
Thomas Strub
  • 1,275
  • 7
  • 20