-1

I need help figuring out how to output the end result of this python to a csv (C:/temp/test.csv) file. Write now it prints it to the prompt. I tried using write, but I kept getting errors no matter what I tried. My main goal is to output the information to the csv without the html included and to have commas separating each result (result2, result3, result4, result5). But right now i would just be happy with being able to output it the way it is to a csv file. Then I can work on the rest.

from selenium import webdriver
import time 
from bs4 import BeautifulSoup
import csv

driver = webdriver.Chrome('C:/temp/chromedriver_win32/chromedriver.exe')

driver.get('https://www.dell.com/support/home/en-us/product-support/product/precision-15-5520-laptop/drivers')

time.sleep(3)

element = driver.find_element_by_xpath("//button[contains(.,'Show all')]").click();

page = driver.page_source

driver.close()

soup = BeautifulSoup(page,'html.parser')

results = soup.find(id='downloads-table')

results2 = results.find_all(class_='dl-desk-view')
results3 = results.find_all(class_='details-control sorting_1')
results4 = results.find_all(class_='details-control')
results5 = results.find_all(class_='btn-download-lg btn btn-sm no-break text-decoration-none dellmetrics-driverdownloads btn-outline-primary')

open('C:/temp/Precision_5520.csv', "w").close

with open('C:/temp/Precision_5520.csv', "a") as csvfile:
    writer = csv.writer(csvfile)
    for r2, r3, r4, r5 in zip(results2, results3, results4, results5):
        writer.writerow([results2, results3, results4, results5])
Testerfy
  • 1
  • 2
  • What errors are you getting? Also have you tried using `Pandas`? – Sabito stands with Ukraine Oct 30 '20 at 03:44
  • I added above what i am currently trying, it is not working because there are too many arguements, i am not sure how to make it write to file the way that it currently outputs to the console. Although that is also to just output it to a txt file. I would prefer not to use pandas, if possible. – Testerfy Oct 31 '20 at 02:08
  • Change `for results2, results3, results4, results5 in zip(results2, results3, results4, results5):` to `for r2, r3, r4, r5 in zip(results2, results3, results4, results5):` What you have currently done is that once this loop runs for the first time, results2/3/4/5 stop being lists and become single values. – Sabito stands with Ukraine Oct 31 '20 at 03:53
  • OK, thanks, that does output to the file, now i just need to look into how to get it so it adds everything, right now it just adds 1 row. – Testerfy Oct 31 '20 at 15:33
  • I think i need to change w to a, heh, nope that still only adds the last item to the next row, How would i get it to output each individual driver's info into a seperate row? – Testerfy Oct 31 '20 at 15:35
  • Can you update the question with your code up till this point? Basically show me the code you are running.. – Sabito stands with Ukraine Oct 31 '20 at 15:47
  • Ok, i added it, to the question – Testerfy Oct 31 '20 at 16:03
  • `writer.writerow([results2, results3, results4, results5])` -> `writer.writerow([r2, r3, r4, r5])` – Sabito stands with Ukraine Oct 31 '20 at 16:04
  • Yes! that worked, I hope you dont mind, I want to use you for one more thing, there are empty rows between the filled ones, do you happen to know what is causing that? – Testerfy Oct 31 '20 at 16:10
  • There is a very good chance that `results2/3/4/5` have `\n` in them. Try printing `print(results2)` and checking... If that is infact true then you need to loop over the values in `results2/3/4/5` and replace `\n` with `''` for all strings. [see](https://stackoverflow.com/a/9347456/11573842) – Sabito stands with Ukraine Oct 31 '20 at 16:28

2 Answers2

0

Here is what i have so far, still have some kinks to work out, but i got my answer from Yatin about getting everything to show up in csv file:

from selenium import webdriver
import time 
from bs4 import BeautifulSoup
import csv
import tkinter as tk
from tkinter import filedialog

root = tk.Tk()
root.withdraw()

file_path = filedialog.askopenfilename()

with open(file_path, 'r') as f:
   list = f.readlines()

for items in list:
   driver = webdriver.Chrome('C:/temp/chromedriver_win32/chromedriver.exe')
   driver.get(items)
   time.sleep(3)
   element = driver.find_element_by_xpath("//button[contains(.,'Show all')]").click();
   page = driver.page_source
   driver.close()
   soup = BeautifulSoup(page,'html.parser')
   results = soup.find(id='downloads-table')
   results2 = results.find_all(class_='dl-desk-view')
   results3 = results.find_all(class_='details-control sorting_1')
   results4 = results.find_all(class_='details-control')
   results5 = results.find_all(class_='btn-download-lg btn btn-sm no-break text-decoration-none dellmetrics-driverdownloads btn-outline-primary')
   new_name = items.replace('https://www.dell.com/support/home/en-us/product-support/product/','').replace('/drivers','').replace('\n','')
   open('C:/temp/' + new_name + '.csv', "w").close   
   with open('C:/temp/' + new_name + '.csv', "a") as csvfile:
      writer = csv.writer(csvfile)
      for r2, r3, r4, r5 in zip(results2, results3, results4, results5):
          writer.writerow([r2, r3, r4, r5])

Will mark as answer when i can, not sure why someone gave a negative one to my question but did not say why : /

Testerfy
  • 1
  • 2
-2

Try iterating over your zipped list and write each line to the csv file. You can add a header row by writing it to the file outside the loop.

alpha32
  • 37
  • 6