0

NEW UPDATE This is the main window. It consists of displaying a green icon (scraping performed correctly) or a red icon (scraping error) next to each text, based on the outcome of the scraping. Then basically, there is an icon with a question mark that is replaced by the red or green icon. In addition to the icon change, the error message is also displayed (the error messages with the exception are contained in another external file, which I use for scraping). Scraping is performed thanks to two py scripts (or rather thanks to their functions) that are imported. They are msg1 = Scraping_Nome_Campionati.scraping_nome_campionati_e_tor, and msg2 = Scraping_Nome_Squadre_MIO.scraping_nome_squadre_e_tor()

def do_scraping():
    
    msg1 = Scraping_Nome_Campionati.scraping_nome_campionati_e_tor()
    if msg1:
        message1.configure(text=msg1)
        message1.configure(foreground="red")
        vuoto_elenco_campionati.config(image=render7)
        
    else:
         vuoto_elenco_campionati.config(image=render8)
         message1.configure(foreground="green")



    msg2 = Scraping_Nome_Squadre_MIO.scraping_nome_squadre_e_tor()
    if msg2:
        message2.configure(text=msg2)
        message2.configure(foreground="red")
        vuoto_elenco_squadre.config(image=render7)

    else:
         vuoto_elenco_squadre.config(image=render8)
         message2.configure(foreground="green")
  
button = Button(test_scraping, text="Avvia", bg='#e95420', foreground='white', command=do_scraping)
button.place(x=116, y=512)

The scraping in the two external files is performed like this (I carry only 1 file for example purposes)

#example
driver.minimize_window()
driver.get("web site")
Element1=driver.find_element_by_class_name("name class")
Element1_text = Element1.text

#insert in database
con = sqlite3.connect('/dababase')
cursor = con.cursor()
records_added_Risultati = 0

    Values = ((Element1_text,), (Element2_text,))
    sqlite_insert_query = 'INSERT INTO xxxxx VALUES (?);'
    count = cursor.executemany(sqlite_insert_query, Values)
    con.commit()

#The error messages are contained in this external scraping file, 
# and NOT in the main window with icons

except NoSuchElementException:
    return "FAILED (NoSuchElementException)"

except NameError:
    return "FAILED (Name Error)"

except ValueError:
    return "FAILED (ValueError)"

if records_added_Risultati == 0:
   return "FAILED: 0 record scraping"

Campionati_per_controllo_errori = [Element1_text, Element2_text]
if any(Campionati_per_controllo_errori):
    return "FAILED (manca 1 campionato)"

PROBLEM: So to recap, the problem is that when scraping is done correctly (both scraping and database inserting), I still get the red error icon + error message. I set a type of error that makes sure that "if a scraping, even 1 only, is not performed correctly, the error must be printed", that is, if 19 of 20 texts are downloaded correctly, an error must appear and its message. The problem is that: even if you scrape 20 texts and everything is done correctly, I get the message that there is an error and a text has not been scraped.

OLD UPDATE

How can I code the following:

  1. "If one or more" of these elements is not detected, then I get a warning message. By warning message, I mean in case there is a scraping problem that results in, NOT an error, but just an empty result. So that one Element is not scraped, but the others following do.

  2. If all are not detected, then do I display a warning message?

So I would need 2 types of code.

#example
driver.minimize_window()
driver.get("web site")
Element1=driver.find_element_by_class_name("name class")
Element1 = Element1text


if Element1 or Element2 or Element3 or Element4 or Element5 = no scraping:
    return "error"

if Element1, Element2, Element3, Element4, Element5 = no scraping:
    return "error"

UPDATE Elements are data scraped with Selenium and saved in a database. For example they are like this:

    #Element1
    driver.get("site")

    for Element1 in driver.find_elements(By.CSS_SELECTOR, xxxxxx'][class^='rxxxxxx']"):
        Element1_text = Element1.text
        count = cursor.execute(sqlite_insert_query, (Element1_text,))
        print(Element1_text)
    driver.close

#insert in database
con = sqlite3.connect('/dababase')
cursor = con.cursor()
records_added_Risultati = 0

    Values = ((Element1_text,), (Element2_text,))
    sqlite_insert_query = 'INSERT INTO xxxxx VALUES (?);'
    count = cursor.executemany(sqlite_insert_query, Values)
    con.commit()

3 Answers3

1

For python3, consider using the any() and all() functions.

From the documentation:

all(iterable)

Return True if all elements of the iterable are true (or if the iterable is empty).

any(iterable)

Return True if any element of the iterable is true. If the iterable is empty, return False.

Now you just need to create an iterable out of your elements, which turns out to be really simple:

some_iterable = [Element1, Element2, Element3, Element4, Element5]
if any(some_iterable):  # Like using or on all of them!
    return "error"

if all(some_iterable):  # Like using and on all of them!
    return "error"

Now we have one final problem! The elements need to have an inherent truth value, they need to be evaluated as either True or False. As @Prophet pointed out in his answer, using driver.find_elements_by_class_name instead of driver.find_element_by_class_name (the difference is in ELEMENTS instead of a single ELEMENT) achieves this, since it provides an empty array (evaluates to False) if the element with that class name was not found.

If one instance of an element of that class was found, you'll be left with an array including only that element, which you can use as follows: Element1[0]. If more than one element is found, as expected, you'll have an array of them, so you need to handle them accordingly.

Edit: More explanation.

any(some_iterable) is equivalent to: Element1 or Element2 or Element3 or Element4 or Element5 here. all(some_iterable) is equivalent to: Element1 and Element2 and Element3 and Element4 and Element5

Now you may be asking yourself why Element1 and Element2 and Element3 and Element4 and Element5 even works! The answer is, if the elements are [] or None, they evaluate to False under the operators (and) and (or), causing the entire expression to evaluate to False. You can read more about False in an if statement here.

You can think of if any(some_iterable) as saying "If any of the elements in some_iterable exists and is non-zero" to some extent, and if any(some_iterable) as saying "If all of the elements in some_iterable exist and all of them are non-zero".

In reality, it's more complex than that (a lot of things return false, not just None, [] and 0), but you get the point.


Updated answer to the updated question.

You want to throw an error when any/all are NOT present. You're missing a not in your code.

Just change:

if any(Campionati_per_controllo_errori):  # If any of them is present, throw an error. Not what we want.
    return "FAILED (manca 1 campionato)"

to:

if not all(Campionati_per_controllo_errori):  # If not all of them are present, throw an error.
    return "FAILED (manca 1 campionato)"
mmdts
  • 757
  • 6
  • 16
  • 1
    Thanks I'm not sure I understand correctly (as I am just starting out with Python). After passing I have to write for example "error message 1"? I get the corresponding error message in case one of the two If conditions above is desired. Can you explain the code better to me please? Don't get angry and forgive me. I'm starting out with Python. thank you – James D. Com Jul 31 '21 at 01:34
  • Changed it to make it clearer. "pass" is just a python keyword that does absolutely nothing. I put it there so that you can replace it with whatever you want. Added more explanation. – mmdts Jul 31 '21 at 01:54
  • Excuse me if I answer now. I don't log in often. I tried as you said. I'm still getting the error message error, despite the scraping being successful. I do not understand. I also checked in the database: all ok, all scraped and saved. Also in the Python console everything is ok the scraping. But I still get the error message. How come? – James D. Com Aug 03 '21 at 16:41
  • Can you share your exact code? I can't help on an issue if I can't see it. – mmdts Aug 04 '21 at 12:11
  • OK thanks. I updated the question, starting from the beginning. I have reported the code of the various scripts (the main window where I would like to display icons (error or successful) and relative error message. I also reported how I perform the scraping with external files that contain the error messages. Thanks You are very kind. I appreciate so much – James D. Com Aug 04 '21 at 16:32
  • Edited my answer to include an update. You should look up booleans and logic operators when you have the time. – mmdts Aug 06 '21 at 17:42
  • Thank you so much for the reply. You're kind. But that's not exactly what he was looking for. I was looking for 2 things: either that everything was not being scraped, or that: "If an element is not being scraped, then there is the error message". For example, if out of 10 elements, 9 are scraped correctly and 1 is not scraped. In this case what would the code look like? Perhaps due to problems of the Google translator I have not expressed myself well above. I really appreciate your help. Thanks and sorry – James D. Com Aug 08 '21 at 13:59
  • Aaaaaa ok. I had misunderstood. I thought that "If not all" "if there are 0 elements", but no, it means "if only one (1) or more elements is missing", right? So taking 10 elements as an example, only 9 are scraped and 1 is not scraped by mistake, this corresponds to the code "If not all" ? – James D. Com Aug 08 '21 at 23:25
  • Yes. Exactly. If not all are present = if any is missing. – mmdts Aug 09 '21 at 01:17
0

You can get all these elements in this manner:

element1=driver.find_elements_by_class_name("name class")

In this case if element was found element1 is non-empty list, otherwise if element was not found this will give you and empty list.
Empty list is False in Python while non-empty list is interpreted as True.
So the first if can be done as:

if element1 and element2 and element3 and element4 and element5:
    return "error1"

And the second if is:

if element1 or element2 or element3 or element4 or element5:
    return "error2"
Prophet
  • 32,350
  • 22
  • 54
  • 79
  • So if, for example, if element1 is an empty list, and element2, 3, 4 and 5 are complete lists, i will have a: NO, YES, YES, YES, YES. In this case the code to use is your second one? (the one with or) – James D. Com Jul 18 '21 at 07:32
  • It is not YES or NO rather Boolean True or False. – Prophet Jul 18 '21 at 07:36
  • When you have several Booleans, the AND between all of them will return True only if ALL the variables are True, otherwise False. But if you doing OR between the Booleans it will return True if at least one variable is True and False only if ALL the variables are False. – Prophet Jul 18 '21 at 07:38
  • I'm sorry if I'm just writing now, but I voted without having tested the code: it seemed correct, but there is a problem. Basically I wrote the code with the various OR. All elements are scraped off correctly, but my script reports error2. I do not understand. The message should not come out. I follow the scraping in the Python console and everything is ok. Is there anything you can add to the code with OR? – James D. Com Jul 19 '21 at 20:19
  • I guess there is at least 1 list is empty here. I hope you wrote the code with indentation, if so no way `error2` is printed unless at least 1 list is empty. So please recheck your result, print the length of each list i.e. `print(len(element1))` and like this for all the 5 lists. – Prophet Jul 19 '21 at 20:38
  • I still find an error. I do not know why. I've updated the question to include more details on what Element1, Element2, etc. are. Maybe you need to write something more specific. If you look at the application update you make me happy. I hope for your help. Thank you – James D. Com Jul 31 '21 at 00:34
-1

the simplest way to check if an element is present is using the "in" keyword. Example:

a = "YES"
b = "NO"
c = "YES"

if "NO" in (a, b, c):
    print("someone is NO")
Miguel Prz
  • 13,718
  • 29
  • 42
  • How can I set the meaning of NO and YES? It seems that Propheta's answer suits me more, but if you put more details in your answer I am grateful. Thank you – James D. Com Jul 18 '21 at 07:35