5

I am using the Python Elasticsearch API to interact with ES in my application. Currently, as soon as the app gets a user request, it does esclient = Elasticsearch("127.0.0.1") and then uses this esclient to search for required data. But I recently read that ES has persistent connections. So,

  1. Should I save the esclient somewhere and reuse it? If yes, how do I do that?
  2. Would there be some resource leak if I forget about esclient after the request and open a new one next time? If yes, how do I cure this/ close the open connection?

Also, same questions for the memcached/ Redis python APIs. I do client = Client("127.0.0.1") multiple times during each user request.

Sorry, might seem like a stupid question but I am a bit confused.

Thanks-in-advance!

yogk
  • 271
  • 5
  • 15

1 Answers1

2

For elasticsearch python client you should just have one instance and keep reusing it.

If you create new ones there will be some connections left hanging, but also both your app and elasticsearch itself will be paying the overhead of creating and maintaining additional connections, which is not good.

If it's a Django or Flask application, you can create an elasticsearch specific resource file and create a connection as follows and reuse it everywhere.

ESCLIENT = Elasticsearch()

Import ESCLIENT everywhere and reuse.

Akhil Mathew
  • 1,571
  • 3
  • 34
  • 69
  • What happens if the connection of ESCLIENT breaks? Do you have to restart the application? Or will it automatically try to reconnect on the next request? Talking specifically about Django. – Andy Feb 22 '23 at 07:48
  • 1
    @Andy The Elasticsearch client in Python provides a number of options for handling connection failures and reconnecting to the Elasticsearch cluster. You can use the max_retries parameter to specify the maximum number of times the client should attempt to reconnect before giving up. Additionally, you can use the retry_on_timeout parameter to specify whether the client should retry requests that time out due to connection issues. In addition to using these built-in options, you could use a retry loop to attempt to reconnect to the Elasticsearch cluster a certain number of times with a delay – Akhil Mathew Feb 22 '23 at 09:22