0

I have been trying to run a python script in NiFi's ExecuteScript processor. Though the catch here is that I don't have server file location access and all the python libraries are installed at "/data/jython", "/data/jython/Lib/site-packages/" and "data/nltk"

Below is the import section of my python script:

import json, traceback, pycountry, requests, geocoder, re, sys, nltk
from nltk.sentiment.vader import SentimentIntensityAnalyzer
from nltk.corpus import stopwords
from java.nio.charset import StandardCharsets
from org.apache.commons.io import IOUtils
from org.apache.nifi.processor.io import StreamCallback
from org.python.core.util import StringUtil

I have added path reference to the packages/libraries: enter image description here

Heres the screenshot of the error message: enter image description here

Is there something I am missing? I have referred to another answer here, but couldn't figure out whats wrong with my code.

atulshgl
  • 55
  • 10

2 Answers2

2

As the other answers state, Apache NiFi's ExecuteScript processor uses Jython, not Python. There is a limitation on the Jython library that it cannot handle native modules (modules that end in .so or are compiled C code, etc.). It is very likely that the pycountry module contains some native module. You can try a work-around proposed by Matt Burgess on the NiFi Developers Mailing List here.

Andy
  • 13,916
  • 1
  • 36
  • 78
  • I tried using sys.path.append() to add library paths. Also tried adding paths to module directory, but it didn't work. – atulshgl Aug 04 '17 at 16:50
  • As @Andy says above, some of the modules you are using are using natively compiled code. See mattyb's answer [here](https://stackoverflow.com/questions/40719469/import-modules-in-nifi-executescript/40722026#40722026) – B. Griffiths Sep 14 '17 at 19:12
0

ExecuteScript processor uses its own Jython Engine to execute your python scripts. As the libraries which you are importing are not available in NIFI inbuild Jython Engine its throwing error.

SOLUTION:

If python is already installed on our machine with all those libraries (the same machine where your NIFI is installed) you can use that python engine to execute your script. you can execute your python code using ExecuteProcess processor. see the configuration of ExecuteProcess.

Safin Biswas
  • 41
  • 1