How to use NiFi ExecuteScript processor with Python?

Question

I have a very basic setup of the ExecuteScript processor in Apache NiFi with a simple Python script (saved as a .py file) as shown here. In the Properties of the processor, I set the Script Engine to python and Script File to the path of this script.

import time

count = 0

while(count < 20):
    print "The counter says: ", count
    count = count + 1
    time.sleep(.1)

And this is the dataflow diagram I made:

I don't see anything outputted to the log or the PutFile. However, I do see the print statements appear in \nifi-0.6.1\logs\nifi-bootstrap.log. My knowledge of this is currently limited. I would appreciate answers from anyone who knows how to use the ExecuteScript processor, or even give me a better example than my current setup.

I wonder if you tried the ExecuteProcess. It should grab the stdout and route it as a flowfile: https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi.processors.standard.ExecuteProcess/index.html — andrew, May 05 '16 at 18:30

score 17 · Answer 1 · answered May 04 '16 at 23:56

Given your script, I think everything is functioning as expected. The script is not producing any FlowFiles which is why nothing is moving from ExecuteScript to the other processors, and anything sent to system out is captured in the bootstrap.log so that is why the print statement ends up there.

Script executing with in ExecuteScript get access to a few standard objects:

session
context
log
REL_FAILURE
REL_SUCCESS

In order to produce FlowFiles you would need call session.create() and take the resulting FlowFile and transfer it to REL_SUCCESS.

The best source of info on the scripting processors is Matt Burgess's blog, this page has some good background (uses Groovy):

http://funnifi.blogspot.com/2016/02/executescript-processor-hello-world.html

This one has a Jython example:

http://funnifi.blogspot.com/2016/03/executescript-json-to-json-revisited_14.html

Not sure what you mean "get access" to standard objects. Does this mean that don't need any special `import` statements in the .py file to reference these? — Michael M, Dec 17 '19 at 22:35
That's exactly what he means, they're global variables, so your script has access to them without having to define them elsewhere. — Eden Trainor, Jan 15 '20 at 19:50

How to use NiFi ExecuteScript processor with Python?

1 Answers1