3

I have a few carefully configured conda environments created using Anaconda. I have written a Python package that uses some of the packages and libraries installed in these environments. Now I would like to run the tests with each test file being executed for each environment to make sure the package can handle working with different Python versions and different set of 3rd party packages available.

Using a tox did not work for me because it tries to recreate those environments locally to run tests against. This is how my tox.ini file looks:

[tox]
envlist = py36, py27
skipsdist = True

[testenv]
basepython = 
    py36: C:\Users\user\AppData\Local\Continuum\Anaconda2\envs\Env1\python.exe
    py27: C:\Users\user\AppData\Local\Continuum\Anaconda2\envs\Env2\python.exe

commands = 
    {envpython} -m unittest discover

However, it is not possible to do just plain pip install into the environments tox creates on each run mainly because there are some compiled libraries and workarounds involved (which are solved in a conda environment - so I have to use those as they are).

I currently have a dummy .bat file with the following content:

"C:\Users\user\AppData\Local\Continuum\Anaconda2\envs\Env1\python.exe" -m unittest discover
"C:\Users\user\AppData\Local\Continuum\Anaconda2\envs\Env2\python.exe" -m unittest discover

which I execute in a Windows cmd to see the results. It does work, but I wonder whether there is any more Pythonic way to run these tests such using py.test or tox. I do not want to recreate the conda environments I already have; I just want to use different Python interpreters accessed at different locations to run my tests.

Alex Tereshenkov
  • 3,340
  • 8
  • 36
  • 61

1 Answers1

2

You can look into pytest-xdist plugin. It allows you to run tests across different python environments.

Usage:

pytest --dist=each --tx popen//python=python3.5 --tx popen//python=python2.7
SilentGuy
  • 1,867
  • 17
  • 27
  • 1
    Thanks a ton, great tool. Where will this plugin read the paths to the Python installations? From tox.ini files? Or is there any other way to define those? The docs https://github.com/pytest-dev/pytest-xdist#running-tests-in-a-python-subprocess do not seem to tell anything on this... – Alex Tereshenkov Oct 17 '18 at 21:05
  • It does not require tox.ini file. Python2 and Python3 should be present in the environment and all the dependencies must be installed in both python2 and python3. Having both python2 and python3 in a virtual environment may be tricky. I would suggest you to try the command without python virtual environment. These commands seem to run tests for me. >python2 --version >python3 --version >pip2 install -r requirements.txt >pip3 install -r requirements.txt >pytest --dist=each --tx popen//python=python3 --tx popen//python=python2 – SilentGuy Oct 18 '18 at 16:08
  • 1
    thanks for getting back. How do I specify paths to conda environments? Could you elaborate on what do you mean by `Python2 and Python3 should be present in the environment `? Thanks! – Alex Tereshenkov Oct 18 '18 at 19:49
  • I don't have experience with conda. I use virtualenv and virtualenvwrapper for python env management. I use pip for package management. Conda may be handling packages and envs differently. – SilentGuy Oct 18 '18 at 20:00
  • OK, I see. So, when you write `python=python3.5` how does `pytest` know where to search for `python.exe` file for this particular environment? – Alex Tereshenkov Oct 18 '18 at 20:27
  • If python3.5 exists in system path, it will run fine. – SilentGuy Oct 18 '18 at 21:40
  • Use this answer to create multiple env https://stackoverflow.com/a/30555944/2312300 – SilentGuy Oct 18 '18 at 21:41