0

I am running NiFi in a container. How do I best install python modules within this container? I need to install pandas. I logged in to the container and installed pandas using pip. But weird thing is, this installed only pandas and numpy. And it has not installed any other dependencies!

I get error when I run ExecuteScript processor, which complains about missing dependencies.

Why is it behaving so differently!

Arun Nalpet
  • 1,190
  • 1
  • 9
  • 20

1 Answers1

1

ExecuteScript uses Jython and does not support Python native dependencies (of which pandas is one). If you want to use native libraries, use ExecuteStreamCommand to invoke the python runtime via the command line, external of the NiFi JVM. See this answer for more details.

Andy
  • 13,916
  • 1
  • 36
  • 78
  • Thanks Andy. I should be able to work on that alternative approach using ExecuteStreamCommand. – Arun Nalpet Apr 25 '20 at 06:18
  • Related question. How do I know what all Jython packages are present that are visible to NiFi? What are default packages that are shipped? – Arun Nalpet Apr 25 '20 at 06:20
  • Can I add more Jython packages if needed? Is it available to download and install it within NiFi? Sorry for flooding with questions! I seem to be on right track now. – Arun Nalpet Apr 25 '20 at 06:22
  • 1
    NiFi doesn't ship with any Python packages. The availability of Jython packages will depend on the operating system distribution you select, the version of JRE present, and any deployment configuration (e.g. Ansible, Terraform, Puppet, etc.) scripts. You can read more here: https://www.jython.org/jython-old-sites/docs/library/indexprogress.html and https://jython.readthedocs.io/en/latest/ModulesPackages/. – Andy Apr 27 '20 at 19:33