Transfer or Extract Computed Data from One Python File to Another

Question

AA computes a function using a software called CasADi. I have installed CasADi, and AA can run on my desktop.

BB has to run in a container and imports data from AA.Because of this, BB cannot run because it doesn't recognize CasADi

How can I change the import statement in file BB to extract data from AA, which is in a different location. Both AA and BB are python files

I have tried the following import statements. Both did not work.

from AA import c_A

from home.user.Desktop.AA import c_A

Do the host and the container share any disk directories? If not, then you need to treat them as two separate computers. You'll have to use something like a socket to send the data. — Tim Roberts, Jul 05 '23 at 19:21
What are you sending? Is it just a numeric array, or some more complicated data structure? — Tim Roberts, Jul 05 '23 at 23:06
No, so I extracted the data into a csv file and moved that csv file to the container. That works better — NAA, Jul 17 '23 at 13:27

score 1 · Answer 1 · answered Jul 07 '23 at 05:11

As long as you have a way to copy files into the container, it's not too hard. In your AA script, you can do:

import numpy as np
...
# Assume the numpy array is in variable "array".
np.save( "transfer.npy", array )

Now, you would have to take a manual to copy the file into the container. Once in the container, in your BB script, you can do:

import numpy as np
...
array = numpy.load( "transfer.npy" )

and that will work.

If you don't have a way to copy files, then you would have to use a TCP client/server thing. That takes more coordination, because one side has to be a server, which runs FIRST and sits and waits for the other side to check in. After they're connected, you can use np.save to save the file to a io.BytesIO object, which acts just like a file, then send the contents over the socket. There are many examples on the web of simple TCP socket exchanges in Python.

Mark Setchell · Answer 2 · 2023-07-07T08:50:03.707

Similar to @TimRoberts answer, but showing how to bind mount and share a directory on your desktop with your docker instance...

In AA, save the array on your Desktop using np.save("/home/YOURUSER/Desktop/transfer.npy", data).

Then start BB in docker, but using a "bind mount" of your Desktop to a folder called /data in docker:

docker run -v "$HOME/Desktop:/data" ...

Then in BB, load the file with:

import numpy as np
na = np.load("/data/transfer.npy")

If you wanted a bit more of a dynamic/interactive interchange between the two processes without involving disk files, you could spin up another docker container with the standard Redis image in it, then you could easily exchange commands/data/Numpy arrays via Redis - see example here.

Transfer or Extract Computed Data from One Python File to Another

2 Answers2