Im using streamlit to create a webapp using python. They host and provision servers for your web app code pulling data from your GitHub repo, it always updates based on what you push to your repo. They also have a "manage app" section which lets me see the logs of their Debian server but no typing or root access.
I have a web app that visualizes data and has a button that runs another .py that scrapes data from a website using selenium. I am looking to have the scraper work when deploying to their website.
Currently, when clicking the button on my webapp through the logs I get an error:
"Message: Service /usr/bin/chromium unexpectedly exited. Status code was: -5"
I have chromium installed and am using its binary path as the driver. I have tried using a downloaded chromedriver (linuxpath) but I get a similar error.
###############################Paths####################
cwd = os.getcwd()
driver_dir = cwd + "\chromedriver.exe"
path = pathlib.Path(__file__).parent / 'chromedriver.exe'
linuxpath = pathlib.Path(__file__).parent / 'chromedriver'
linuxbinarypath = '/usr/bin/chromium'
#########################################################
options = Options()
#options.add_argument('--no-sandbox') # fixes (unknown error: DevToolsActivePort file doesn't exist)
options.add_argument('--headless')
#options.add_argument('--disable-extensions')
#options.add_argument('--log-level=3')
#########################################################
try:
if platform.system()=='Windows': # when running locally to test the webapp
options.add_argument('--disable-gpu')
browser = webdriver.Chrome(path, options=options) # Chrome Driver Windows Path --if running on windows
else: # when deploying to streamlit 'Debian/linux'
try:
options.binary_location = str(linuxbinarypath)
except Exception as e:
print(e)
browser = webdriver.Chrome(linuxbinarypath, options=options) # Chrome Driver Linux Path --if running on linux (Streamlit Debian Deployment)
If anyone has any tips please let me know. Again this use-case is a little different because I'm using a THEIR own hosting to do something slightly autonomous. When I run this app locally, my computer pings the website to scrape the data. So in this case, I am having their server ping the website and scrape the data. People said it has been done using SeleniumBase, but there isn't much info on it. All of my paths are correct and I have been spending about 3 weeks on this one error. If anyone has tips let me know. My backup plan is to ditch streamlit hosting and just deploy this through a docker container elsewhere. (If anyone has suggestions on other methods of deploying this app for free let me know if this is tricky)
