Need Wikipedia web scraper to continuously ask for user input

Question

I need the below code to ask for user input again, after executing and showing results. I guess a while loop would be best but not sure how to do it as have BeautifulSoup and requests library in use.

Any help would be greatly appreciated.

from bs4 import BeautifulSoup

user_input = input("Enter article:")

response = requests.get("https://en.wikipedia.org/wiki/" + user_input)
soup = BeautifulSoup(response.text, "html.parser")

list = []
count = 0

IGNORE = ["Wikipedia:", "Category:", "Template:", "Template talk:", "User:",
               "User talk:", "Module:", "Help:", "File:", "Portal:", "#", "About this", ".ogg", "disambiguation", "Edit section"]

for tag in soup.select('div.mw-parser-output a:not(.infobox  a)'):
    if count <= 10:
        title = tag.get("title", "")
        if not any(x in title for x in IGNORE) and title != "":
            count = count + 1
            print(title)
            list.append(title)
    else:
        break

Depends on your setup, but you could try running `aioconsole`'s input() function to set some variable. Then in your while loop where the scraper happens, you can just create some check on that input variable and maybe reset it after checking it the first time — Diego Cuadros, May 16 '22 at 19:43
`I need the below code to ask for user input again, after executing and showing results.` So did you try that? Just locate in your code where the result is shown and ask for user input there? — Melon, May 17 '22 at 08:01

score 0 · Answer 1 · answered May 16 '22 at 21:35

Use function with return statement

Example

import requests
from bs4 import BeautifulSoup

IGNORE = ["Wikipedia:", "Category:", "Template:", "Template talk:", "User:",
          "User talk:", "Module:", "Help:", "File:", "Portal:", "#", "About this", ".ogg", "disambiguation",
          "Edit section"]


def get_user_input():
    user_input = input("Enter article:")
    if len(str(user_input)) > 0:
        return get_response(user_input)
    else:
        return get_user_input()


def get_response(user_input):
    response = requests.get("https://en.wikipedia.org/wiki/" + user_input)
    soup = BeautifulSoup(response.text, "html.parser")

    title_list = []
    count = 0

    for tag in soup.select('div.mw-parser-output a:not(.infobox  a)'):
        if count <= 10:
            title = tag.get("title", "")
            if not any(x in title for x in IGNORE) and title != "":
                count = count + 1
                print(title)
                title_list.append(title)
                print(title_list)
        else:
            return get_user_input()


if __name__ == '__main__':
    get_user_input()

Need Wikipedia web scraper to continuously ask for user input

1 Answers1