0

I'm tryinig to call a script from another one. I need to pass a dictionary to the script I call which will contain some matrices.

The solution I found is to pass through a pickle save in order to reload it in the called script.

I'm wondering if what I did is the only solution or is it possible to pass directly the dictionary to the called script?

Here the minimal example:

file1 (where the script is called):

import subprocess
import sys
import os
import pickle

if __name__ == '__main__':
    dictionary = {'a':1,'b':2}
    filename_temp = os.path.abspath('filename.pickle')
    with open(filename_temp, 'wb') as handle:
        pickle.dump(dictionary, handle, protocol=pickle.HIGHEST_PROTOCOL)

    subprocess.Popen([sys.executable, os.path.join("", "Subproc.py "), filename_temp], shell=True)

file 2 (the script being called):

import sys
import pickle

class useDict():
    def __init__(self,filename_temp=None):
        if filename_temp:
            with open(filename_temp, 'rb') as handle:
                Dictionary = pickle.load(handle)
                self.Dictionary = Dictionary
        else:
            self.Dictionary = None
        print(self.Dictionary)

def main(filename_temp=None):
    print(filename_temp)
    useDict(filename_temp)

if __name__ == '__main__':
    if len(sys.argv)>1:
        main(sys.argv[1])
ymmx
  • 4,769
  • 5
  • 32
  • 64
  • I hope it helps https://stackoverflow.com/questions/18006161/how-to-pass-dictionary-as-command-line-argument-to-python-script – planet260 Oct 10 '19 at 11:20
  • Better if we pass json string by `json.dumps` as argument and `json.loads` in second script – Yugandhar Chaudhari Oct 10 '19 at 11:21
  • 1
    You really need to call the other script with subprocess? Maybe you can import script 2.main with an import statement in script 1? Then you can pass a dictionary directly. – Kvothe Oct 10 '19 at 11:21
  • my issue is the dictionary is really big, it contains 100*10^6 matrices. Please @Yugandhar Chaudhari, could you explain more about the Json dumps? – ymmx Oct 10 '19 at 11:27
  • if I use the Json.dumps I get TypeError: Object of type 'ndarray' is not JSON serializable. So I don't think I can use this if I have matrices – ymmx Oct 10 '19 at 11:30
  • `json.dumps` or `pickle.dump` wont make any difference in your case as I can see you are passing is file name. For that large data what you are doing makes sense – Yugandhar Chaudhari Oct 10 '19 at 11:54
  • @ymmx why not just do `import subproc` also note that numpy has its own [binary format](https://docs.scipy.org/doc/numpy/reference/generated/numpy.save.html) that would probably be faster than pickle and certainly much faster that anything you could do with JSON. – Sam Mason Oct 10 '19 at 14:20
  • 1
    I don't know a priori from which script I will call the subproc module. I want to be able to select the module from a Qt interface. That's why I use a subprocess. – ymmx Oct 10 '19 at 14:42

3 Answers3

1

It's depend on your usage but here is three options.

Option 1

You add your dictionary logic in the same file as the one that need it

Option 2

You still have your dictionary in another file so you do

#file 1

 dictionary = {'a':1,'b':2}

And then in file 2 :

#file 2
from file1 import dictionary

use_dict(dictionary)

Option 3

You generate the dictionary and use it later, or several times, and you don't want to generate it again. Here you can do as you do now. But as mentioned in comment, prefer json format as it human readable and safer.

#file 1
import json

dictionary = {'a':1,'b':2} 
filename_temp = os.path.abspath('filename.pjson') 
with open(filename_temp, 'w') as handle:
json.dump(dictionary, handle) 

#file 2
import json

 with open(filename_temp, 'r') as handle:
     self.Dictionary = json.load(handle)

Edit 1

OP mention in the comment that he need to pass an array. Here also, there is three possibilities.

Option 1

Store variables needed to produce the ndarray in the dictionary.

Option 2*

Convert the ndarray to nested list and store it in json, then retrieve the nested list and convert it to array. This will be my preferred option.

numpy tolist

Option 3

Use pickle instead of json as mentioned in previous option 3.

Florian Bernard
  • 2,561
  • 1
  • 9
  • 22
  • Ok. But Json cannot contain ndarray, so I will be limited to serializable object. – ymmx Oct 10 '19 at 11:38
  • OK, so three possibility here also, you add variables needed to produce your ndarray in the dictionary, you convert your array to an list, you use pickle. – Florian Bernard Oct 10 '19 at 12:14
0

Why do you want to generate the dictionary in file1, save it to disk and then load the dictionary back in file2? That increases the script run time a lot. I would suggest a better way is:

  1. Generate the dictionary in file1
  2. Modify file2 so that file2 contains only the function definitons that are needed to operate on the dictionary.
  3. Import the defined functions present in file2 inside file1 itself
  4. After import, use the imported function on the dictionary in file1 itself.
Karthick Mohanraj
  • 1,565
  • 2
  • 13
  • 28
-1

You can use commanline arguments to pass your dictionary

import os

mydictionary = {'1':'One', '2':'Two', '3':'Three'}

os.system(f"python myscript.py \"{mydictionary}\"")

This is your main script

import ast, sys
data = sys.argv[1]
mydict = ast.literal_eval(data)
print(mydict)
print(mydict["1"])
print(mydict["2"])
print(mydict["3"])

That's how you can read the dictionary

ikibir
  • 456
  • 4
  • 12