6

I've desperetaly tried to set a pytest pipeline CI/CD for my personal projet hosted by gitlab.

I tried to set up a simple project with two basic files:

file test_core.py, witout any other dependencies for the sake of simplicity:

# coding: utf-8
# !/usr/bin/python3

import pytest
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.firefox.options import Options

def test_basic_headless_selenium_example():
    """Test selenium installation by opening python website.
    (inspired by https://selenium-python.readthedocs.io/getting-started.html)
    """
    opts = Options()
    opts.headless = True
    driver = webdriver.Firefox(options=opts)
    driver.get("http://www.python.org")
    driver.close()

File .gitlab-ci.yml, for CI/CD automatic tests:

stages:
  - tests

pytest:python3.7:
  image: python:3.7
  stage: tests
  services:
    - selenium/standalone-firefox:latest
  script:
 #   - apt-get update && apt-get upgrade --assume-yes
    - wget -O ~/FirefoxSetup.tar.bz2 "https://download.mozilla.org/?product=firefox-latest&os=linux64"
    - tar xjf ~/FirefoxSetup.tar.bz2 -C /opt/
    - ln -s /opt/firefox/firefox /usr/lib/firefox
    - export PATH=$PATH:/opt/firefox/
    - wget -O ~/geckodriver.tar.gz "https://github.com/mozilla/geckodriver/releases/download/v0.28.0/geckodriver-v0.28.0-linux64.tar.gz"
    - tar -zxvf ~/geckodriver.tar.gz -C /opt/
    - export PATH=$PATH:/opt/
    - pip install selenium pytest
    - pytest

On my laptop, the pytestcommand works fine 100% of time. When I push a commit to gitlab, I deseperately get errors:

>       raise exception_class(message, screen, stacktrace)
E       selenium.common.exceptions.WebDriverException: Message: Process unexpectedly closed with status 255
/usr/local/lib/python3.7/site-packages/selenium/webdriver/remote/errorhandler.py:242: WebDriverException
=========================== short test summary info ============================
FAILED test_selenium.py::test_basic_headless_selenium_example - selenium.comm...
============================== 1 failed in 1.29s ===============================
Cleaning up file based variables
00:01
ERROR: Job failed: exit code 1

I've created a simple project: https://gitlab.com/OlivierLuG/selenium_firefox that reproduce this example. The failed pipeline can be directely found here : https://gitlab.com/OlivierLuG/selenium_firefox/-/pipelines/225711127

Does anybody have a clue how to fix this error ?

pyOliv
  • 1,253
  • 1
  • 6
  • 21
  • Either install `Xvfb` and use `xvfb-run pytest`, or configure firefoxdriver to run in headless mode. – hoefling Dec 05 '20 at 17:49
  • Thanks, the pytest is already in headless mode. I've managed to get the project to work. I'll post an answer. – pyOliv Dec 05 '20 at 19:00

2 Answers2

5

I've finally managed to ping gitlab CI on green with the below .gitlab-ci.yml file.

Note that I'm not a fan of yaml language. To make the file shorter, I've used a shared block of code, named install_firefox_geckodriver. Then, I've configured 2 jobs with python 3.7 and 3.8, that call this block. The keys to make this kind of test to work are: _ run in headless mode (this was already the case for me) _ install firefox and geckodriver with command lines _ install firefox dependencies _ use gitlab selenium service

Here is my yaml file. The sucessful pipeline can be found here : https://gitlab.com/OlivierLuG/selenium_firefox/-/pipelines/225756742

stages:
  - tests

.install_firefox_geckodriver: &install_firefox_geckodriver
  - apt-get update && apt-get upgrade --assume-yes
  - apt-get install gconf-service libasound2 libatk1.0-0 libc6 libcairo2 libcups2 libdbus-1-3 libexpat1 libfontconfig1 libgcc1 libgconf-2-4 libgdk-pixbuf2.0-0 libglib2.0-0 libgtk-3-0 libnspr4 libpango-1.0-0 libpangocairo-1.0-0 libstdc++6 libx11-6 libx11-xcb1 libxcb1 libxcomposite1 libxcursor1 libxdamage1 libxext6 libxfixes3 libxi6 libxrandr2 libxrender1 libxss1 libxtst6 ca-certificates fonts-liberation libappindicator1 libnss3 lsb-release xdg-utils --assume-yes
  - wget -nv -O ~/FirefoxSetup.tar.bz2 "https://download.mozilla.org/?product=firefox-latest&os=linux64"
  - tar xjf ~/FirefoxSetup.tar.bz2 -C /opt/
  - ln -s /opt/firefox/firefox /usr/lib/firefox
  - export PATH=$PATH:/opt/firefox/
  - wget -nv -O ~/geckodriver.tar.gz "https://github.com/mozilla/geckodriver/releases/download/v0.28.0/geckodriver-v0.28.0-linux64.tar.gz"
  - tar -zxvf ~/geckodriver.tar.gz -C /opt/
  - export PATH=$PATH:/opt/


pytest:python3.7:
  image: python:3.7
  stage: tests
  services:
    - selenium/standalone-firefox:latest
  script:
    - *install_firefox_geckodriver
    - pip install selenium pytest
    - pytest

pytest:python3.8:
  image: python:3.8
  stage: tests
  services:
    - selenium/standalone-firefox:latest
  script:
    - *install_firefox_geckodriver
    - pip install selenium pytest
    - pytest
pyOliv
  • 1,253
  • 1
  • 6
  • 21
  • Nice, but got `tar (child): bzip2: Cannot exec: No such file or directory` because bzip2 was missing in my Alpine-Linux based image. Added that to `apt-get install ...` packages and it worked. – Jan Jun 23 '22 at 14:57
0

If you are using Ubuntu based images, the installation procedure can be simplified to bellow, allow getting the last geckodriver version:

before_script:
  # Configura the timezone, needed for some libraries.
  # Solution from https://dev.to/setevoy/docker-configure-tzdata-and-timezone-during-build-20bk
  - TZ=America/Sao_Paulo # Numeric inputs: 2 - 134.
  - ln -snf /usr/share/zoneinfo/$TZ /etc/localtime
  - echo $TZ > /etc/timezone
  # Add the needed libraries.
  - apt update && apt -y upgrade


.install_firefox_geckodriver: &install_firefox_geckodriver
  - apt -y install firefox  curl wget # It needs timezone configuration made in "before_script"
  #- wget -nv -O ~/FirefoxSetup.tar.bz2 "https://download.mozilla.org/?product=firefox-latest&os=linux64"
  - wget $(curl -s https://api.github.com/repos/mozilla/geckodriver/releases/latest | grep 'browser_' | cut -d\" -f4 | grep linux64.tar.gz$)
  - tar -xf gecko*
hildogjr
  • 754
  • 2
  • 6
  • 17