-3

I am using scrapy for web scraping, and I want to store data to csv files. How do I obtain the current url so that I can use it to name my csv files? Using python 2.7.14, scrapy 1.5. Does scrapy provide any such functionality?

OmK
  • 147
  • 2
  • 9

2 Answers2

1

You can get the get URL by "response.request.URL" . here I'm get URL in scrapy shell

    >>> fetch("http://www.bing.com/")
    2018-04-05 15:53:00 [scrapy.core.engine] DEBUG: Crawled (200) <GET http://www.bing.com/> (referer: None)
    >>> response
    <200 http://www.bing.com/>
    >>> response.request.url
    'http://www.bing.com/'
    >>>
Svickie7
  • 32
  • 8
-1

This

self.request.url

or this (if you need the query_string)

self.request.query_string
Gareth O'Connor
  • 60
  • 8