0

This is the follow up of Import Modules in Nifi ExecuteScript

I am new to python as well as nifi. I am trying to execute my python script in ExecuteScript processor.

I want to access a server. so i used paramiko client. But when i run the processor, it shows "Import error No module named constant_time" at line session.write(). Though i have this constant_time.py under "/usr/local/lib/python2.7/dist-packages/ "

enter image description here

I have also the path "/usr/local/lib/python2.7/dist-packages/ " in sys.path. I have also given this path in the "Module Directory" property.

This is my code:

import json, pysftp, paramiko
import java.io
from org.apache.commons.io import IOUtils
from java.nio.charset import StandardCharsets
from org.apache.nifi.processor.io import StreamCallback

class ModJSON(StreamCallback):
  def __init__(self):
    pass
  def process(self, inputStream, outputStream):

    text = IOUtils.toString(inputStream, StandardCharsets.UTF_8)
    inputText = text.rstrip('\r\n')
    json_content = json.loads(inputText)
    body = ''
    try:
       body = json_content['id']['body']
       body_encoded = body.encode('utf-8')
    except (KeyError,TypeError,ValueError):
       body_encoded = ''

    ssh_client = paramiko.SSHClient()
    ssh_client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
    ssh_client.load_system_host_keys()
    ssh_client.connect('server', username='xxx', password='xxxx')

    sftp_client = ssh_client.open_sftp()
    text_file = sftp_client.open ('/doc/body.txt', 'w')   
    text_file.write("%s"%body_encoded)
    text_file.close()
    outputStream.write(bytearray(json.dumps(body, indent=4).encode('utf-8')))

flowFile = session.get()
if (flowFile != None):
   flowFile = session.write(flowFile, ModJSON())
   flowFile = session.putAttribute(flowFile, "filename", flowFile.getAttribute('filename').split('.')[0]+'_translated.json')
   session.transfer(flowFile, REL_SUCCESS)

Any help would be appreciated.

Community
  • 1
  • 1
vishnu
  • 891
  • 2
  • 11
  • 20
  • Possible duplicate of [Import Modules in Nifi ExecuteScript](http://stackoverflow.com/questions/40719469/import-modules-in-nifi-executescript) – Andy Nov 22 '16 at 21:25

1 Answers1

0

As explained by Matt Burgess in this answer, the ExecuteScript processor uses Jython, not Python, so compiled modules are not available. There is ongoing discussion of how/if to add this feature.

Community
  • 1
  • 1
Andy
  • 13,916
  • 1
  • 36
  • 78
  • yeah. In that discussion, it was explained how to import modules. It was said to add the lib path in "Module Directory " property in ExecuteScript Processor. I also did that adding in the "module Directory". But still it shows the error. so do you mean that i can never import such modules? – vishnu Nov 22 '16 at 22:07
  • I believe that is the case currently -- as Matt wrote: "According to [this SO post](http://stackoverflow.com/questions/29814603/python-import-error-cannot-import-module-named-counter-which-is-a-so-file), paramiko uses Crypto which has native libraries, so cannot be used in Jython (see the bottom of [this post](https://funnifi.blogspot.com/2016/03/executescript-json-to-json-revisited_14.html) for my comment on that)." – Andy Nov 22 '16 at 22:09
  • Oh!! Thanks Andy. So, is there any other module , that i can import for accessing a server? or what is the other way around for accessing and working with server in ExecuteScript? – vishnu Nov 22 '16 at 22:18
  • You have a couple options. You can try to find a Python module that does not include or depend on compiled modules, which you could use in Jython. I am unaware if any exist. You could also switch to Groovy (I'm 100% certain) or Ruby (70% certain) for the `ExecuteScript` processor, as both allow module inclusion. I cannot predict if/when NiFi Jython will accept compiled modules. – Andy Nov 22 '16 at 22:57
  • Thanks Andy let me try. – vishnu Nov 23 '16 at 08:10