Compiling scikit-image on Windows to run tests

Question

I'm stumped. I'm developing some enhancements to scikit-image which are failing the automated build tests, probably due to rounding errors. I therefore need to get the automated tests running on my Windows system so that I can debug and work out what's wrong. I've so far tried two approaches, neither of which are working:

1. In my Anaconda Python 3.6 environment, when I try to run the automated tests, I am getting the following error:

   RuntimeError: module compiled against API version 0xc but this version of numpy is 0xb

   ...which I have found reference to in other contexts, but have not been able to eliminate.

2. Since the automated test do run (but fail) on a Python 3.5-based system, I thought things might work if I tried a local Python 3.5 environment. Here, I am running into the issue that, despite being installed, the environment cannot find the MS C++ compiler cl.exe. It is installed in C:\Program Files (x86)\Microsoft Visual Studio\2017\BuildTools\VC\Tools\MSVC\14.15.26726\bin\HostX86\x64\ and is found and executed by my Python 3.6 environment, but my Python 3.5 environment doesn't find it despite me adding that directory to my PATH. I should add that my Python 3.6 environment finds it without the directory being added to the PATH. I understand that both Python 3.5 and 3.6 use MSVC 14.0.

I would prefer to fix the problem in my Python 3.6 environment if possible. Any assistance much appreciated.

Update

I have made a box-fresh Python 3.6 conda environment as follows:

conda create --name sk36 python=3.6
conda activate sk36
conda install scikit-image --only-deps
conda install cython
git clone https://github.com/scikit-image/scikit-image.git
cd scikit-image
pip install -e .
pytest skimage/feature

The specific error I am getting is as follows:

..\Anaconda3\lib\site-packages\py\_path\local.py:662: in pyimport
    __import__(modname)
skimage\__init__.py:135: in <module>
    from .data import data_dir
skimage\data\__init__.py:13: in <module>
    from ..io import imread, use_plugin
skimage\io\__init__.py:7: in <module>
    from .manage_plugins import *
skimage\io\manage_plugins.py:24: in <module>
    from .collection import imread_collection_wrapper
skimage\io\collection.py:12: in <module>
    from ..external.tifffile import TiffFile
skimage\external\tifffile\__init__.py:1: in <module>
    from .tifffile import imsave, imread, imshow, TiffFile, TiffWriter, TiffSequence
skimage\external\tifffile\tifffile.py:292: in <module>
    from . import _tifffile
E   RuntimeError: module compiled against API version 0xc but this version of numpy is 0xb

...which appears to have something to do with tifffile. Since this package wasn't originally explicitly installed in my new environment, I tried installing various versions of it, including some which downgraded numpy and scipy. Still the same error as above.

Having done some more research it would appear that something is seeing numpy 1.13.x when in fact version 1.15.4 is installed. Here is the full output from conda list:

# Name                    Version                   Build  Channel
blas                      1.0                         mkl    anaconda
ca-certificates           2018.03.07                    0    anaconda
certifi                   2018.10.15               py36_0    anaconda
cloudpickle               0.6.1                    py36_0    anaconda
cycler                    0.10.0           py36h009560c_0    anaconda
cython                    0.29             py36ha925a31_0    anaconda
dask-core                 0.20.0                   py36_0    anaconda
decorator                 4.3.0                    py36_0    anaconda
freetype                  2.9.1                ha9979f8_1    anaconda
icc_rt                    2017.0.4             h97af966_0    anaconda
icu                       58.2                 ha66f8fd_1    anaconda
imageio                   2.4.1                    py36_0    anaconda
intel-openmp              2019.0                      118    anaconda
jpeg                      9b                   hb83a4c4_2    anaconda
kiwisolver                1.0.1            py36h6538335_0    anaconda
libpng                    1.6.35               h2a8f88b_0    anaconda
libtiff                   4.0.9                h36446d0_2    anaconda
matplotlib                3.0.1            py36hc8f65d3_0    anaconda
mkl                       2019.0                      118    anaconda
mkl_fft                   1.0.6            py36hdbbee80_0    anaconda
mkl_random                1.0.1            py36h77b88f5_1    anaconda
networkx                  2.2                      py36_1    anaconda
numpy                     1.15.4           py36ha559c80_0    anaconda
numpy-base                1.15.4           py36h8128ebf_0    anaconda
olefile                   0.46                     py36_0    anaconda
openssl                   1.0.2p               hfa6e2cd_0    anaconda
package_has_been_revoked  1.0                           0    enable_revoked
pillow                    5.3.0            py36hdc69c19_0    anaconda
pip                       18.1                     py36_0    anaconda
pyparsing                 2.3.0                    py36_0    anaconda
pyqt                      5.9.2            py36h6538335_2    anaconda
python                    3.6.7                h33f27b4_1    anaconda
python-dateutil           2.7.5                    py36_0    anaconda
pytz                      2018.7                   py36_0    anaconda
pywavelets                1.0.1            py36h8c2d366_0    anaconda
qt                        5.9.6            vc14h1e9a669_2    anaconda
scikit-image              0.15.dev0                 <pip>
scipy                     1.1.0            py36h4f6bf74_1    anaconda
setuptools                40.5.0                   py36_0    anaconda
sip                       4.19.8           py36h6538335_0    anaconda
six                       1.11.0                   py36_1    anaconda
sqlite                    3.25.2               hfa6e2cd_0    anaconda
tifffile                  0.15.1          py36h452e1ab_1001    conda-forge
tk                        8.6.8                hfa6e2cd_0    anaconda
toolz                     0.9.0                    py36_0    anaconda
tornado                   5.1.1            py36hfa6e2cd_0    anaconda
vc                        14.1                 h21ff451_3    anaconda
vs2015_runtime            15.5.2                        3    anaconda
wheel                     0.32.2                   py36_0    anaconda
wincertstore              0.2              py36h7fe50ca_0    anaconda
zlib                      1.2.11               h8395fce_2    anaconda

Update 2

I've solved the problem for Python 3.6, and I think there's enough information above for the astute to be able to work out what was wrong. I'll put the solution in an answer below.

A cleanly built Python 3.5 environment can't find the compiler, so that issue still remains.

Answer 1

One approach you could try is to upgrade your numpy with

pip install numpy --upgrade

as described here: RuntimeError: module compiled against API version a but this version of numpy is 9

Otherwise (if for some reason you cannot upgrade numpy) I would suggest going with a virtual environment for scikit-image project. I just tried it on Windows 10 and was able to successfully execute tests. My steps (from cmd, inside the project folder):

• conda uninstall scikit-image to remove any previously built/installed versions
• conda -n scikit-image python=3.6 to create a virtual environment for this project (I used python 3.6, but you can change it to 3.5)
• activate scikit-image activated the new virtual env
• pip install -r requirements.txt -- installed dependencies (without this step I wasn't getting the dependencies for tests installed)
• pip install -e .
• pytest

Answer 2

It turns out that pytest wasn't actually installed in the correct environment, it was being invoked from base which did indeed have numpy 1.13.3 installed. Installing it in the cleanly built Python 3.6 environment solved the problem for Python 3.6 at least.