Accessing python class variable defined inside the main module of a script

Question

I have a django project that uses celery for async task processing. I am using python 2.7.

I have a class in a module client.py in my django project:

# client.py
class Client:
    def __init__(self):
        # code for opening a persistent connection and saving the connection client in a class variable
        ...
        self.client = <connection client>

    def get_connection_client(self):
        return self.client

    def send_message(self, message):
        # --- Not the exact code but this is the function I need to access to for which I need access to the client variable---
        self.client.send(message)
    # Other functions that use the above method to send messages
    ...

This class needs to be instantiated only once to create one persistent connection to a remote server.

I run a script connection.py that runs indefinitely:

# connection.py
from client import Client
if __name__ == '__main__':
    clientobj = Client()
    client = clientobj.get_connection_client()
    # Blocking process
    while True:
        # waits for a message from the remote server
        ...

I need to access the variable client from another module tasks.py (needed for celery).

---

# tasks.py
...
from client import Client
@app.task
def function():
    # Need access to the client variable
    # <??? How do I get an access to the client variable for the 
    # already established connection???>
    message = "Message to send to the server using the established connection"
    client.send_message(message)

All the three python modules are on the same machine. The connection.py is executed as a standalone script and is executed first. The method function() in tasks.py is called multiple times across other modules of the project whenever required, thus, I can't instantiate the Client class inside this method. Global variables don't work.

---

In java, we can create global static variable and access it throughout the project. How do we do this in python?

Approaches I can think of but not sure if they can be done in python:

• Save this variable in a common file such that it is accessible in other modules in my project?
• Save this client as a setting in either django or celery and access this setting in the required module?
• Based on suggestions by sebastian, another way is to share variables between running processes. I essentially want to do that. How do I do this in python?

For those interested to know why this is required, please see this question. It explains the complete system design and the various components involved.

I am open to suggestions that needs a change in the code structure as well.

Answer 1

multiprocessing provides all the tools you need to do this.

connection.py

from multiprocessing.managers import BaseManager
from client import Client()
client = Client()
class ClientManager(BaseManager): pass
ClientManager.register('get_client', callable=lambda: client)
manager = ClientManager(address=('', 50000), authkey='abracadabra')
server = manager.get_server()
server.serve_forever()

tasks.py

from multiprocessing.managers import BaseManager
class ClientManager(BaseManager): pass
ClientManager.register('get_client')
manager = ClientManager(address=('localhost', 50000), authkey='abracadabra')
manager.connect()
client = manager.get_client()

@app.task
def function():
    message = "Message to send to the server using the established connection"
    client.send_message(message)

Answer 2

I dont have experience working with django, but if they are executed from the same script you could make the Client a singleton, or maybe declaring the Client in the init.py and then import it wherever you need it.

If you go for the singleton, you can make a decorator for that:

def singleton(cls):
    instances = {}

    def get_instance(*args, **kwargs):
        if cls not in instances:
            instances[cls] = cls(*args, **kwargs)
        return instances[cls]

    return get_instance

Then you would define:

# client.py

@singleton
class Client:
    def __init__(self):
        # code for opening a persistent connection and saving the connection client in a class variable
        ...
        self.client = <connection client>

    def get_connection_client(self):
        return self.client

Thats all I can suggest with the little description you have given. Maybe try to explain a little better how everything is run or the parts that are involved.

Answer 3

Python has class attributes (attributes that are shared amongst instances) and class methods (methods that act on the class itself). Both are readable on either the class and an instance.

# client.py
class Client(object):
    _client = None

    @classmethod
    def connect(cls):
        # dont do anything if already connected
        if cls._client is None:
            return

        # code for opening a persistent connection and saving the connection client in a class variable
        ...
        cls._client = <connection client>


    @classmethod
    def get_connection_client(cls):
        return cls._client


    def __init__(self):
        # make sure we try to have a connection on initialisation
        self.connect()

Now I'm not sure this is the best solution to your problem.

Answer 4

If connection.py is importing tasks.py, you can do it in your tasks.py:

import __main__ # connection.py
main_globals = __main__.__dict__ # this "is" what you getting in connection.py when you write globals()
client = main_globals["client"]  # this client has the same id with client in connection.py

BaseManager is also an answer but it uses socket networking on localhost and it is not a good way of accessing a variable if you dont already using multiprocessing. I mean if you need to use multiprocessing, you should use BaseManager. But if you dont need multiprocessing, it is not a good option to use multiprocessing. My code is just taking pointer of "client" variable in connection.py from interpreter.

Also if you want to use multiprocessing, my code won't work because the interpreters in different processes are different.

Answer 5

Use pickle when reading it from file.