1

My question is similar to this one but I got a slightly different scenario and actual code, I need to call a function giving it 2 params, one of them an array of objects (as seen in the example python2 file code) and receive the output, also an array of objects (not showed in this example code for simplification purposes).

My python2 file: 

#!/usr/bin/env python
from subprocess import call
if __name__ == "__main__":
    pathToFile = '/home/scripts/'
    arr = [{'x': 1, 'y': 2, 'z': 3}, {'x': 2, 'y': 2, 'z': 2}]
    output = call([pathToFile + 'python3', 3, arr])

My python3 file:

#!/usr/bin/env python3
import numpy as np
import sys
def python3Function(poses,kernelCount):
    #complex calculations using py3 functionalities hidden for minified example
    x = 3
    y = 5
    z = np.atleast_1d(x) @ np.atleast_1d(y) 
    return z

if __name__ == "__main__":
    kernelCount= sys.argv[1] 
    poses= sys.argv[2]  
    print(python3Function(poses,kernelCount))

Currently it seems the call method only takes in strings because I get following error:
TypeError: execv() arg 2 must contain only strings
Importing the method directly also doesn't work because it uses python specific syntax like @ and results in the error:
SyntaxError: invalid syntax

The answer from the linked question gives me two options, I quote:

  • Talk over a pipe or socket
  • Enable such python 3 features as you can from __future__ or use a library like six to write code which is compatible with both.

I have absolutely no idea what that guy is talking about and which of these 3 methods I should use in my case...I need actual code for my example, the simpler the better, then I will understand.

Cold_Class
  • 3,214
  • 4
  • 39
  • 82
  • 1
    Convert the array to string, then in python 3 convert str back to list, or you can use pickle to save the array and load the array with pickle in python3 – Stack Jun 17 '18 at 14:03

1 Answers1

1

You can't pass arbitrary objects to a child process given the UNIX process model that the subprocess module implements. You can only pass strings. So you need to serialize (i.e., encode) your object to a string and in the child process unserialize (i.e., decode) it. This is true for most other IPC (inter process communication) mechanisms.

There are many ways to do this. The obvious is using repr() to serialize (i.e., encode) the object. The problem with that obvious solution is that converting the serialized data into an object is not safe because it generally results in the code receiving the serialized data to eval the string. Which means it is a security hole. You should use an encoding that is safe to use in such situations. Such as JSON or Google protobufs.

Kurtis Rader
  • 6,734
  • 13
  • 20
  • I can't accept your answer, unless you can provide some code for my example with your JSON-solution. Otherwise I will post my own working workaround through a csv file soon. – Cold_Class Jul 29 '18 at 08:09
  • @Cold_Class I'll be happy to write code for you. Shall we agree on $50/hour? Seriously, I pointed you in the right direction. All you have to do is a few minutes googling "python json" or just `pydoc json`. – Kurtis Rader Jul 29 '18 at 20:08