0

I have an issue with the code where, only the last cdi data of gene_name and description will be saved. The selenium opens up both cdi's and brings back the gene_name and description. For some reason, only the second cdi is saved into a excel file. Is there any way to save both cdi's of gene_name and description into 1 excel file.

from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
import time
import pandas as pd



path = (r"C:\Users\hogyu\PycharmProjects\Project_Selenium\drivers\chromedriver.exe")

driver = webdriver.Chrome(path)

#B0041.2c, ZC518.3b

cdis = ["ZC518.3b","B0041.2c"]


for cdi in cdis:


        data = {}
        df = pd.DataFrame(columns=['Gene_name', 'Description'])

        dictionary = {'One': 1, 'Two': 2}


        try:
            driver.get("https://wormbase.org/#012-34-5")
            search = driver.find_element_by_id("Search")
            search.send_keys(cdi)
            search.send_keys(Keys.RETURN)

            id = WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.CLASS_NAME, "locus")))
            description = WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.CLASS_NAME, "text-min")))

            data['Gene_name'] = id.text
            data['Description'] = description.text

            df = df.append(data, ignore_index=True)
            df.index += 1


            print(df)

        finally:

            time.sleep(1)


driver.back()
driver.refresh()
time.sleep(2)
driver.quit()

print((df).to_excel(r"C:/Users/hogyu/OneDrive/Desktop/Python codes/experiments/worbase_data.xlsx"))
  • You are performing the file write after the loop finishes, so you're only righting the final value of `df` to the file. If you want all of them written to the file, you'll need to move your call to `to_excel()` inside your loop. – gallen Jul 02 '20 at 18:29
  • so the last command print((df).to_excel(r"C:/Users/hogyu/OneDrive/Desktop/Python codes/experiments/worbase_data.xlsx")) should this be within the loop?? –  Jul 02 '20 at 18:43
  • i am a beginner, can you elaborate on is please. i entered df).to_excel(r"C:/Users/hogyu/OneDrive/Desktop/Python codes/experiments/worbase_data.xlsx")) within the for loop right after df.append() but it still only produces the last value on to the excel –  Jul 02 '20 at 18:53
  • That's because `to_excel()` over-writes the file on each call. [This question](https://stackoverflow.com/questions/47737220/append-dataframe-to-excel-with-pandas) has several answers that will assist you in what you need to do. – gallen Jul 02 '20 at 19:02

0 Answers0