I am using selenium google chrome driver to get some web page in Python, when use the google driver, I want the google driver was exit completely. But there still have chrome process in the kubernetes pod after use the driver, this is my python code:
@staticmethod
def fetch_music_download_url(music_name: str):
chrome_driver_service = Service(ChromeDriverManager(chrome_type=ChromeType.GOOGLE).install())
chrome_options = Options()
chrome_options.add_argument("--no-sandbox")
chrome_options.add_argument("--headless")
chrome_options.add_argument("--disable-gpu")
chrome_options.add_argument("--remote-debugging-port=9230")
driver = webdriver.Chrome(service=chrome_driver_service,
options=chrome_options,
executable_path="/usr/local/bin/chromedriver")
try:
driver.maximize_window()
driver.get('http://tool.liumingye.cn/music/?page=audioPage&type=migu&name=' + music_name)
driver.implicitly_wait(5)
driver.find_element(By.CSS_SELECTOR, ".aplayer-list-download.iconfont.icon-xiazai").click()
urls = [a.get_attribute('href') for a in
driver.execute_script('return document.querySelectorAll(".modal-body a[href*=\'http\']")')]
for url in urls:
if "listenSong.do" in url:
logger.info("fetched url:" + url)
FetchMusic.do_save_music_download_url(url)
except Exception as e:
logger.error("scrapy impl error", e)
finally:
driver.stop_client()
driver.close()
driver.quit()
chrome_driver_service.stop()
this code works fine for a while, after the app run some long time. there still have google chrome process in the kubernetes pods, I do not know where is going wrong, and I already added the close code at the finally block. what should I do to ensure the chrome exists every time used the chrome? Am I missing something with my Python 3 code?
I also have question: why my code did not close the chrome process? I think it always to the same thing, why it did not work as expect? the finally block would always ensure the close method.
