0

I'm trying to figure out what is the best way to use UnboundID LDAP SDK in system with high TPS (30k - 50k). Most I have found around internet suggest to use multi-threading together with operation directly on connection pools (link1, link2), for example inside one thread:

SearchResult searchResult = connectionPool.search("dc=example,dc=com", SearchScope.SUB, "(uid=john.doe)");

But it seems to me that for high TPS this is not doable. If latency from LDAP server is to big you would have to create a large number of threads to sustain high TPS. Another approach is to use Async operations but then you loose benefits of connection pool. Any advice is really appreciated (especially from someone who was in similar situation).

Josip Vujcic
  • 102
  • 1
  • 9
  • Is there an issue creating a pool with a large number of connections? The LDAP SDK under the covers is using asynchronous operations? Are you limited to use a single process only? Are you sure that your tests are not CPU bound on the client side? When working with search load in particular, we found that monitoring the client load is also important as it can become CPU bound. At times we had to use multiple client systems to saturate the server and get the throughput we wanted. We could achieve well over 100k ops/second for search operations. – Bertold Kolics Jan 23 '17 at 14:40
  • As I know UnboundId SDK doesnt use asynchornus operations if you do LDAP operations directly on pool. I can take out connection from pool (or create a new one) and do async operations on one connection but then you lose integrated pool properties. Also, the problem is with latency, for example if a response takes 50ms, that would mean that we need to create about 2500 connections (and threads) to sustain 50000 TPS. – Josip Vujcic Jan 25 '17 at 11:44
  • Also, client is not CPU bound. – Josip Vujcic Jan 25 '17 at 11:46
  • I maybe mistaken but one of the constructors to the LDAPConnectionPool is an LDAPConnection which is used as a template for other connection in the pool. You can create that connection with an LDAPConnectionOption with asynchronous mode enabled. – Bertold Kolics Jan 26 '17 at 14:14
  • Yes, I know, but it looks like pool doesn't use that option. If I have a pool with 2 connections and in 3 parallel threads I call pool.search(...), one of those searches will block until another thread releases connection. I'm not 100% sure but this is what I think happens. – Josip Vujcic Jan 31 '17 at 20:27
  • Have you tried using a thread-local connection pool? – Bertold Kolics Feb 03 '17 at 20:19
  • No, I will have a look at that. Thanks for the hint. – Josip Vujcic Feb 07 '17 at 21:42

0 Answers0