I am facing a strange problem which I could track down to the python logging package. Let me shortly explain what I want to do: the goal is to create html reports with python. I am using the R rmarkdown package which runs python code trough reticulate using a local virtualenv.
The Problem:
As soon as I install the python logging package rmarkdown runs into a problem when loading matplotlib. I have written a small test script to reproduce the example.
My system:
- Ubuntu 18.04 bionic
- Python 2.7.15rc1
Test "1" script (without logging):
- Create a new virtualenv (
venv). - Use
venv/bin/pipto installmatplotlib. - Run
reticulate::import(at the end viarmarkdown::render).
Test "2" script (with logging):
- Create a new virtualenv (
venv). - In addition to the first test: install
loggingviavenv/bin/pip. - Use
venv/bin/pipto installmatplotlib. - Run
reticulate::import(at the end viarmarkdown::render).
The modules installed (virtualenv):
backports.functools-lru-cache 1.5
cycler 0.10.0
kiwisolver 1.0.1
logging 0.4.9.6 <- only for "test 2"
matplotlib 2.2.3
numpy 1.15.1
pip 18.0
pkg-resources 0.0.0
pyparsing 2.2.0
python-dateutil 2.7.3
pytz 2018.5
setuptools 40.2.0
six 1.11.0
subprocess32 3.5.2
wheel 0.31.1
The system site packages do have the same module version.
Results:
All tests from test 1 (without logging) work nicely.
The tests from test 2 (with loging) fail when using the virtualenv. When calling rmarkdown::render (see below), when using the system python installation (not virtualenv) they work nice as well.
There seem to be something strange with reticulate when logging is installed in a virtualenenv.
The output of the test script (see below):
The full output including the error:
----------- no logging package installed ------------
Module(matplotlib)
Module(matplotlib)
--------- with logging package installed ------------
Error in py_module_import(module, convert = convert) :
AttributeError: 'module' object has no attribute 'NullHandler'
Detailed traceback:
File "/home/retos/Downloads/venvtest/venv/lib/python2.7/site-packages/matplotlib/__init__.py", line 168, in <module>
_log.addHandler(logging.NullHandler())
Calls: <Anonymous> -> py_module_import -> .Call
Execution halted
Module(matplotlib)
The Module(matplotlib) output is the success message of loading the module via reticulate::import. As one can see only the one test fails where the virtualenv is used with installed logging python module.
Anyone having an idea what could case these problems? I spent quite some time to identify the source of the error, but I am kind of lost now ...
Test script to reproduce the error:
Here is a small bash/shell script to reproduce my tests.
#!/bin/bash
# New virtual environment and install matplotlib
echo " ----------- no logging package installed ------------"
if [ -d venv ] ; then rm -rf venv ; fi
virtualenv venv &>/dev/null > /dev/null
venv/bin/pip install matplotlib > /dev/null
# Print installed packages
Rscript -e "reticulate::use_python('venv/bin/python'); reticulate::import('matplotlib')"
Rscript -e "reticulate::import('matplotlib')"
# New virtual environment and install logging and matplotlib
echo " --------- with logging package installed ------------"
if [ -d venv ] ; then rm -rf venv ; fi
virtualenv venv > /dev/null
venv/bin/pip install logging > /dev/null
venv/bin/pip install matplotlib > /dev/null
# Print installed packages
Rscript -e "reticulate::use_python('venv/bin/python'); reticulate::import('matplotlib')"
Rscript -e "reticulate::import('matplotlib')"
I first thought it is related to the problem "ImportError: cannot import name cbook" but the solution there did not work.
May thanks in advance! R
