Python - webCrawler - driver.close incorrect syntax

Question

Novice programmer, currently making a WebCrawler and came up with driver.close()

^incorrect syntax as shown below,

However, I used driver above with no problem so I'm pretty perplexed at the moment

I appreciate all the help I can get

thanks in advance team

    import sys
    from selenium import webdriver as wd
    # from bs4 import BeautifulSoup as bs
    
    from selenium.webdriver.common.by import By
    from selenium.webdriver.support.ui import WebDriverWait
    from selenium.webdriver.support import expected_conditions as EC
    # import pymysql as my
    import time
    # from Tour import TourInfo
    
    
    # First Url =
    # https://stamprally.org/?search_keywords=&search_keywords_operator=and&search_cat1=145&search_cat2=0
    
    # Second Url =
    # https://stamprally.org/?search_keywords=&search_keywords_operator=and&search_cat1=66&search_cat2=0
    
    
    driver.get(main_url)
    
    
    
    try:
        element = WebDriverWait(driver, 10).until(
            # 지정한 요소 한개가 발견되면 웨이트 종료
            EC.presence_of_element_located((By.ID, 'header_search_cat1'))
        )
    except Exception as e:
        print('오류 발생', e)
    
    driver.implicitly_wait(10)
    
    for prefectureValue in range(66, 121):
        offshorePrefectureValue = 145
        try:
            driver.get(
                f"https://stamprally.org/?search_keywords=&search_keywords_operator=and&search_cat1={prefectureValue}&search_cat2=0)")
            print(driver.current_url)
    
    # close  off
        driver.close()
        driver.quit()
        sys.exit()

SyntaxError: invalid syntax
PS C:\Users\eong\Desktop\stamprally_crawl> python run.py
  File "C:\Users\eong\Desktop\stamprally_crawl\run.py", line 70
    driver.close()
    ^
SyntaxError: invalid syntax

score 1 · Answer 1 · answered Apr 15 '21 at 10:53

1

In case you opened single window only you have nothing to driver.quit() from after performing driver.close()

answered Apr 15 '21 at 10:53

Prophet

32,350
22
54
79

thanks for the input. I'll take that into consideration. Though it still does not solve the syntax error. this is a true mind duming moment for me – noh Apr 15 '21 at 11:10
Could you share the syntax error you receiving inside your question? – Prophet Apr 15 '21 at 11:12
I would also say that you are trying to quit/close the driver in loop i.e several times while you initialized it only once in the beginning however I do not think this is a syntax error. – Prophet Apr 15 '21 at 11:24
i'm curious if you are also getting the same syntax error, thanks alot for the help really appreciating it – noh Apr 15 '21 at 11:30
I even do not have python installed on my PC today :) – Prophet Apr 15 '21 at 11:37

vitaliis · Accepted Answer · 2021-04-15T12:26:27.673

1

Remove this from loop (indentation problem). I think the is the main mistake.

driver.close()
driver.quit()
sys.exit()

It quits driver after the first cycle in your loop. Ream here about the difference between quit() and close(): Difference between webdriver.Dispose(), .Close() and .Quit() Additionally, put break if your conditions becomes correct. Also, you declare offshorePrefectureValue but do not really use it. Another problem does not relate to your question:

try:
    element = WebDriverWait(driver, 10).until(
        # 지정한 요소 한개가 발견되면 웨이트 종료
        EC.presence_of_element_located((By.ID, 'header_search_cat1'))
    )
except Exception as e:
    print('오류 발생', e)

You do not need try/except here as well. Selenium will throw exception if an element is not present.

edited Apr 15 '21 at 12:26

answered Apr 15 '21 at 12:20

vitaliis

4,082
5
18
40

Valid points thanks @vitaliis I changed my code as below, ''' for prefectureValue in list(range(66, 121))+[145]: try: driver.get( f"https://stamprally.org/?search_keywords=&search_keywords_operator=and&search_cat1={prefectureValue}&search_cat2=0)") print(driver.current_url) except Exception as e: print('오류 발생', e) ''' Best, feel free to let me know if you see anything off cheers mate – noh Apr 15 '21 at 14:33
You can also check here how to use explicit wait in Selenium https://stackoverflow.com/questions/67082774/hidden-phone-number-cant-be-scraped/67082898#67082898 You will need it later for sure – vitaliis Apr 15 '21 at 14:55

score 0 · Answer 3 · answered Apr 15 '21 at 12:15

A try needs to have a except (or finally) block as well. So either remove the try, or add an except to catch the error.

Cant believe i spent so much time looking for syntax when i missed the bloody catchException code thinking that i do not need exception catch because i know this would not error out

Python - webCrawler - driver.close incorrect syntax

3 Answers3