2

I am using Dataiku DSS v6.0 which uses pandas==0.23 and this cannot be manually upgraded. I am also trying to use modin==0.6, which is only compatible with pandas==0.25. I have tried using modin==0.3 which requires pandas==0.23, but this older version is throwing exceptions with some of the pandas methods I am calling.

Is there any workaround to this? Can I somehow force modin==0.6 to accept pandas==0.23? Otherwise is there a way in which I can install both pandas==0.23 and pandas==0.25 and force DSS to use 0.23 and modin to use 0.25?

JBE
  • 11,917
  • 7
  • 49
  • 51
KOB
  • 4,084
  • 9
  • 44
  • 88
  • Is this helpful [https://github.com/mitsuhiko/multiversion](https://github.com/mitsuhiko/multiversion)? – Mihail Feraru Feb 04 '20 at 07:53
  • This package is built for Python 2.x, and I am using 3.6.4 and so pip3 cannot find the package. Also, I wouldn't trust using a package that hasn't been updated in 9 years! – KOB Feb 04 '20 at 08:16
  • 1
    I am unable to find any relevant solution. This [article](https://medium.com/knerd/the-nine-circles-of-python-dependency-hell-481d53e3e025) talks about this kind of problem. I will research the topic and maybe I'll code a solution myself. – Mihail Feraru Feb 04 '20 at 10:06
  • It might be possible, but as far as I know, it's going to be difficult. Mitsuhiko's _multiversion_ is an interesting approach. You could also try and see how far you get with this: ["Multi-version installs" from _PyPA_'s _Python Packaging User Guide_](https://packaging.python.org/guides/multi-version-installs/). Otherwise by hacking some custom importers with [_importlib_](https://docs.python.org/3/library/importlib.html), some examples here: https://pymotw.com/3/sys/imports.html#custom-importers – sinoroc Feb 04 '20 at 10:27

2 Answers2

0

Dataiku is provided with a concept of code environments. by using this feature you can have a separated set of dependancies for your own code or for other specific operations.

leldo
  • 386
  • 2
  • 9
0

Starting with DSS 8.0.0 Pandas 1.0 is supported

However, recents versions of modin rely on more recents version of pandas; modin==0.8.0 allows pandas==1.0.5 so that the most recent version of modin compatible with DSS as of today

dmn_nbk
  • 1
  • 1