0

I have the following for loop:

    for titles in titles:
        title = titles.xpath("a/text()").extract()
        link = titles.xpath("a/@href").extract()
        print(title, link)

How can I dump the title and link into a formmated .csv file?

john doe
  • 2,233
  • 7
  • 37
  • 58

4 Answers4

3

You should use the python CSV module. Look here for more information: Writing List of Strings to Excel CSV File in Python.

Here is an example for your problem:

import csv
results = []

# your code... also add this to your for loop.
    results.append([title, link])

csv_file = open("csv_file.csv",'wb')
wr = csv.writer(csv_file)
for row in results:
    wr.writerow(row)
Community
  • 1
  • 1
Preston Hager
  • 1,514
  • 18
  • 33
  • Thanks for the help preston!... when I tried this approach the csv file is blank... I got the following exception:`File "/Users/user/PycharmProjects/scrapy_tests/ttest/spiders/test.py", line 28, in parse wr.writerow(row) TypeError: a bytes-like object is required, not 'str` – john doe Oct 04 '16 at 16:41
  • 2
    Change `'wb'` to `'w'`, can't see why you opened in binary mode. – Dimitris Fasarakis Hilliard Oct 04 '16 at 16:43
  • 2
    You can use either I think, the examples I found used binary writing so that's what I used. – Preston Hager Oct 04 '16 at 18:25
2

Something like this (documentation): 

...
doc_path = "path/to/doc.csv" # must exist, first folder on the place
                             # where you are running the script!
with open(doc_path, 'a') as doc:
    for entry in titles:
        title = entry.xpath("a/text()").extract()
        link  = entry.xpath("a/@href").extract()
        print(title, link)
        doc.write(str(title)+','+str(link)+'\n')
berna1111
  • 1,811
  • 1
  • 18
  • 23
  • 1
    Thanks berna!... what about something with pandas? – john doe Oct 04 '16 at 16:47
  • 1
    You could use something like [`pandas.DataFrame.to_csv`](http://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.to_csv.html), but even people using pandas use traditional `write` ([example](http://stackoverflow.com/questions/15017072/pandas-read-csv-and-filter-columns-with-usecols)). – berna1111 Oct 05 '16 at 03:59
1

Use a csv writer.

Basic usage:

import csv
with open('file.csv') as f:
    writer = csv.writer(f)
    writer.writerow(["title", "link"])
Tim D
  • 508
  • 4
  • 4
1
result = []
for titles in titles:
    title = titles.xpath("a/text()").extract()
    link = titles.xpath("a/@href").extract()
    print(title, link)
    result += [title + ',' + link + '\n']

with open("result.csv" "w+") as f:
    f.writelines(result)
python必须死
  • 999
  • 10
  • 12
  • 1
    Thanks!... I got this exception: ` line 24, in parse result += [title + ',' + link + '\n'] TypeError: can only concatenate list (not "str") to list` – john doe Oct 04 '16 at 16:44