0

Following is the structure of my code:

Part 1:

var result =  <-- a very resource heavy calculation-->;

Part 2:

var a = result.x;
var b   = result.y;

print a;
print b;

Now, I want to build a program that allows me to have variable result available (i.e. Part 1) all the time and to run the "Part 2" again and again. So it will be like a program in a running state which has pre-calculated variables and uses it again and again.

Ankit Sharma
  • 47
  • 9
  • Part 2 will be ideally sent to an user when he calls this service. I just want to avoid calculating result variable each time. – Ankit Sharma Nov 03 '15 at 08:07
  • I think you may be looking for "interprocess communication" between basically 1 "server" Python program and n "client" ones. Here is a link that may help: http://stackoverflow.com/questions/6920858/interprocess-communication-in-python ZeroMQ seems like a nice solution. You could also consider using an in-memory database like Redis. – Shashank Nov 03 '15 at 08:41
  • @Shashank How do I create a service, which loads this object once and each time this service is called (`for eg: localhost:8080/xyzservice?action=getx`) it just uses that variable. **STEPS: 1**. ajax call to a server **STEP 2**. Server has running instance of this program which has result variable stored somewhere **STEP 3**. Server responds with a value Now, how exactly do I make a service which runs only a section of code again and not the complete code? – Ankit Sharma Nov 03 '15 at 09:27
  • There are a lot of ways to do it. In that specific example you could just parse the action string and have a corresponding handler function. One way to do this would be to use a Python dictionary where the key are strings and the values are functions e.g. : `actionhandlers = {'getx': }` The handler functions can be defined before you make the dictionary to make the code cleaner. – Shashank Nov 03 '15 at 10:01

1 Answers1

0

This looks like a producer-consumer problem.

Some guy talking about that, in Python: http://agiliq.com/blog/2013/10/producer-consumer-problem-in-python/

Basically, make a producer thread that generates result, then have a consumer thread that reads it. Ensure that access to the result variable is synchronized / serialized. Run both threads simultaneously, to taste.

Jameson
  • 6,400
  • 6
  • 32
  • 53
  • Thanks @Jameson for the quick response. Actually the result variable is a single object and calculating the object again and again with a thread would also be a very resource heavy task. I used pickle to save this object to my drive to have it "always available" and the size of the file was in the ~200Mb range. I want something that allows me to have this variable always available in the RAM itself. – Ankit Sharma Nov 03 '15 at 08:18
  • oops, didn't see the "pre-calculated" part, sorry. Can't you just unpickle the object back into a variable in your program, then? – Jameson Nov 03 '15 at 08:22
  • Yes, I did that. But again, "unpickle" operation on such a huge object takes a lot of time. And supposing I run it as a service which multiple users can access, will the memory requirement be a multiple of 200* ( number of users) ? – Ankit Sharma Nov 03 '15 at 08:28
  • You only have to unpickle it once at service startup, though, right? Into say, a single global variable that would not be copied as new instances of objects (say that handle requests for clients) got created and died off. This being read-only data greatly simplifies the problem. – Jameson Nov 03 '15 at 08:35
  • Yes, that's what I want to know. How do I create a service, which loads this object once and each time this service is called (`for eg: localhost:8080/xyzservice?action=getx`) it just uses that variable. **STEPS: 1**. ajax call to a server **STEP 2**. Server has running instance of this program which has result variable stored somewhere **STEP 3**. Server responds with a value Now, how exactly do I make a service which runs only a section of code again and not the complete code? – Ankit Sharma Nov 03 '15 at 09:25
  • Here is an example of something like that: https://wiki.python.org/moin/BaseHttpServer . Just use the example code to get started and put a global variable in the file with your huge unpickled thing. – Jameson Nov 03 '15 at 09:27
  • Thanks Jameson, your last comment seems to work well. Please add this comment to the final solution. Thanks again. – Ankit Sharma Nov 03 '15 at 11:02