3

What is the best way to go about making this work? Since Selenium doesn't work well with asyncio, the bot will eventually die. It works as intended, but will crash eventually. I do understand what the issue is, especially since selenium is so heavy, but is there a good work around?

I get the error message: Task was destroyed but it is pending!

I don't have anything else, no traceback. I have to reboot the bot (discord) and it'll work again for a bit. 

@bot.command(pass_context=True)
async def weather(ctx, arg):
    if ctx:
        list = ('lf','oahu')
        await bot.say('*Loading webdriver.....*')
        if arg in list:
            if arg == 'lf':
                website = '--'
                name = 'Los Feliz Map'
            if arg == 'oahu':
                website = '--'
                name = 'Oahu Map'
            load_site = webdriver.Chrome()
            # load site on chrome
            load_site.get(website)
            await bot.say('*Loading ' + str(name) + '...*')
            load_site.find_element_by_xpath('//*[@id="main-map"]/div[2]/div[1]/div/a[2]').click()
            load_site.find_element_by_xpath('//*[@id="main-map"]/div[2]/div[2]').click()
            load_site.find_element_by_xpath('//*[@id="main-map"]/div[2]/div[2]/div/form/div[3]/label[6]/div/span').click()
            await asyncio.sleep(2) #sleep to ensure weather loads on site before screenshot
            load_site.save_screenshot('weather.png')
            await bot.say('*Taking screenshot and saving image....*')
            await bot.send_file(ctx.message.channel, 'weather.png')
            load_site.quit()
            print('Succesfully sent image of ' + str(arg) + ' - ' + str(ctx.message.author.name))

I left out the website because it's a private one.

sb2894
  • 95
  • 1
  • 11
  • 4
    I don't think you're going to have much success. Blocking APIs like selenium really don't work in asynchronous applications. Your best bet would be to use a purpose built asynchronous library. [`arsenic`](https://github.com/HDE/arsenic) seems to be roughly that, but you'd have to experiment with it. – Patrick Haugh Feb 21 '18 at 01:47
  • Yeah I found arsenic, definitely gonna have to do some reading on that. Luckily "click" and "screenshot" still work with arsenic, at a quick glance I didn't see anything about searching with xpath though. will look again later tonight, thank you. – sb2894 Feb 21 '18 at 03:21
  • 3
    You can use `run_in_executor` to run blocking code in it's own thread without blocking. You can then run all your code in the thread and return what is needed. See example here: https://stackoverflow.com/questions/53587063/using-subprocess-to-avoid-long-running-task-from-disconnecting-discord-py-bot/53597795#53597795 – Benjin Apr 17 '19 at 09:21
  • You could try using `playwright`, which supports asynchronous usage. – Jacob Lee Apr 22 '21 at 19:31
  • Using a browser is a bad idea (in general), your better off using aiohttp and beautiful soup to webscrape the website – mcdonalds291 May 17 '21 at 01:14

2 Answers2

0

Short answer: its not possible. If you somehow manage to get this to work, it wont work well.

Long answer: Selenium is a blocking API, you can't really use it with asynchronous scripts. If you really want to use a web browser you can try to find an alternative like https://pypi.org/project/aioselenium/.

mcdonalds291
  • 2,024
  • 1
  • 5
  • 14
0

I made this work by using asynchio.to_thread.


def my_blocking_selenium_code(number: int):
    pass

tasks = []
for i in range(5):
    task = asyncio.to_thread(my_blocking_selenium_code, i)
    tasks.append(task)
await asyncio.gather(*tasks)
Mike Reiche
  • 382
  • 3
  • 12