18

I am using the LDAP SDK from this site: https://www.unboundid.com/products/ldap-sdk/ . I would like to make a search operation which returns a lot of entries.

According to the FAQ's site, ( https://www.unboundid.com/products/ldap-sdk/docs/ldapsdk-faq.php#search ) I have to use a SearchResultListener implementation.

So here is what I did:

 public class UpdateThread extends Thread implements SearchResultListener {
 ...
 // create request
 final SearchRequest request = new SearchRequest(this, instance.getBaseDN(),SearchScope.SUB, filter);
 // Setting size limit of results.
 request.setSizeLimit(2000);

 ...

 // Get every result one by one.
 @Override
public void searchEntryReturned(SearchResultEntry arg0) {
    System.out.println("entry "+arg0.getDN());

}

The problem is that "searchEntryReturned" returns a maximum of 1000 results. Even if I set the size limit to "2000".

stage
  • 4,225
  • 4
  • 15
  • 8

4 Answers4

22

It is pretty simple to implement a paged LDAP query using standard java, by using the adding a PagedResultsControl to the LdapContext, without using a third party API as per Neil's answer above.

Hashtable<String, Object> env = new Hashtable<String, Object>(11);
env
    .put(Context.INITIAL_CONTEXT_FACTORY,
        "com.sun.jndi.ldap.LdapCtxFactory");

/* Specify host and port to use for directory service */
env.put(Context.PROVIDER_URL,
    "ldap://localhost:389/ou=People,o=JNDITutorial");

try {
  LdapContext ctx = new InitialLdapContext(env, null);

  // Activate paged results
  int pageSize = 5;
  byte[] cookie = null;
  ctx.setRequestControls(new Control[] { new PagedResultsControl(pageSize,
      Control.NONCRITICAL) });
  int total;

  do {
    /* perform the search */
    NamingEnumeration results = ctx.search("", "(objectclass=*)",
        new SearchControls());

    /* for each entry print out name + all attrs and values */
    while (results != null && results.hasMore()) {
      SearchResult entry = (SearchResult) results.next();
      System.out.println(entry.getName());
    }

    // Examine the paged results control response
    Control[] controls = ctx.getResponseControls();
    if (controls != null) {
      for (int i = 0; i < controls.length; i++) {
        if (controls[i] instanceof PagedResultsResponseControl) {
          PagedResultsResponseControl prrc = (PagedResultsResponseControl) controls[i];
          total = prrc.getResultSize();
          if (total != 0) {
            System.out.println("***************** END-OF-PAGE "
                + "(total : " + total + ") *****************\n");
          } else {
            System.out.println("***************** END-OF-PAGE "
                + "(total: unknown) ***************\n");
          }
          cookie = prrc.getCookie();
        }
      }
    } else {
      System.out.println("No controls were sent from the server");
    }
    // Re-activate paged results
    ctx.setRequestControls(new Control[] { new PagedResultsControl(
        pageSize, cookie, Control.CRITICAL) });

  } while (cookie != null);

  ctx.close();

Example copied from here.

PeterK
  • 1,697
  • 10
  • 20
11

Although it's almost certainly the case that the server is enforcing the size limit of 1000 entries, there are potentially ways to get around that by issuing the request in multiple parts.

If the server supports the use of the simple paged results control (as defined in RFC 2696 and supported in the LDAP SDK as per https://docs.ldap.com/ldap-sdk/docs/javadoc/com/unboundid/ldap/sdk/controls/SimplePagedResultsControl.html), then you can use it to iterate through the results in "pages" containing a specified number of entries.

Alternately, the virtual list view (VLV) request control (https://www.unboundid.com/products/ldap-sdk/docs/javadoc/index.html?com/unboundid/ldap/sdk/controls/VirtualListViewRequestControl.html) could be used, but I would probably only recommend that if the server doesn't support the simple paged results control because the VLV request control also requires that the results be sorted, and that likely either requires special configuration in the server or some pretty expensive processing in order to be able to service the request.

Neil Wilson
  • 1,706
  • 8
  • 4
10

I solved like @PeterK , but with some modifications

    public List<MyUser> listUsers() {
    LOG.info("listUsers() inicio");
    List<MyUser> users = new ArrayList<MyUser>();

    Hashtable env = new Hashtable();
    env.put(Context.INITIAL_CONTEXT_FACTORY, INITIAL_CTX);
    env.put(Context.PROVIDER_URL, 'ldap://192.168.10.10:389');
    env.put(Context.SECURITY_AUTHENTICATION, CONNECTION_TYPE);
    env.put(Context.SECURITY_PRINCIPAL, USER_ADMIN_PASSWORD);
    env.put(Context.SECURITY_CREDENTIALS, USER_ADMIN);

    try {
        LdapContext ctx = new InitialLdapContext(env, null);

        // Activate paged results
        int pageSize = 1000;
        byte[] cookie = null;
        ctx.setRequestControls(new Control[] { new PagedResultsControl(pageSize, Control.NONCRITICAL) });
        int total;

        do {
            /* perform the search */
            SearchControls sc = new SearchControls();
            sc.setSearchScope(SearchControls.SUBTREE_SCOPE);
            String filtro = "(&(sAMAccountName=*)&(objectClass=user))";
            NamingEnumeration results = ctx.search(getBaseDn(ctx), filtro, sc);

            /* for each entry */
            while (results.hasMoreElements()) {
                SearchResult result = (SearchResult) results.nextElement();
                Attributes attributes = result.getAttributes();
                //convert to MyUser class
                MyUser user = toUser(attributes);
                users.add(user);
            }

            // Examine the paged results control response
            Control[] controls = ctx.getResponseControls();
            if (controls != null) {
                for (int i = 0; i < controls.length; i++) {
                    if (controls[i] instanceof PagedResultsResponseControl) {
                        PagedResultsResponseControl prrc = (PagedResultsResponseControl) controls[i];
                        total = prrc.getResultSize();
                        if (total != 0) {
                            System.out.println("***************** END-OF-PAGE " + "(total : " + total + ") *****************\n");
                        } else {
                            System.out.println("***************** END-OF-PAGE " + "(total: unknown) ***************\n");
                        }
                        cookie = prrc.getCookie();
                    }
                }
            } else {
                System.out.println("No controls were sent from the server");
            }
            // Re-activate paged results
            ctx.setRequestControls(new Control[] { new PagedResultsControl(pageSize, cookie, Control.CRITICAL) });

        } while (cookie != null);

        ctx.close();

    } catch (NamingException e) {
        System.err.println("PagedSearch failed.");
        e.printStackTrace();
    } catch (IOException ie) {
        System.err.println("PagedSearch failed.");
        ie.printStackTrace();
    } catch (Exception ie) {
        System.err.println("PagedSearch failed.");
        ie.printStackTrace();
    }

    LOG.info("listUsers() size = " + (users.size()));
    LOG.info("listUsers() fim");

    return users;
}


private MyUser toUser(Attributes attributes) throws NamingException {
    if (attributes != null) {
        String fullName = attributes.get("distinguishedName") != null ? attributes.get("distinguishedName").get().toString() : null;
        String mail = attributes.get("mail") != null ? attributes.get("mail").get().toString() : null;
        String userName = attributes.get("cn") != null ? attributes.get("cn").get().toString() : null;
        String userPrincipalName = attributes.get("userPrincipalName") != null ? attributes.get("userPrincipalName").get().toString() : null;

        if (userPrincipalName != null) {
            String[] user = userPrincipalName.split("@");
            if (user != null && user.length > 0) {
                userName = user[0];
            }
        }

        MyUser user = new MyUser();
        user.setFullName(fullName);
        user.setEmail(mail);
        user.setName(userName);
        user.setUserPrincipalName(userPrincipalName);
        user.setRoles(getRolesUser(attributes));

        return user;
    }

    return null;
}
  • PeterK will never see your message as you did it – Drew Jan 08 '16 at 19:28
  • you would enter a comment under his Answer. Remember: under your Answer the following people are alerted when you post a comment under your Answer ... (1) the Op I think but don't hold me to it, (2) another commenter like myself if that person is the sole person with a comment thread as in no one else is commenting, (3) someone you @theirName that is already under your answer in a comment or the Op – Drew Jan 09 '16 at 19:39
  • It was only because (2) was the case above that I was alerted that you wrote that comment above "How do this?" ... otherwise if some 3rd person commented, I would **not** have been alerted. A lot of people think they are being ignored on SO in comments but in reality they have no clue they are being spoken to due to the way it works – Drew Jan 09 '16 at 19:41
  • Thanks @Drew , but my posted code above, does not fit in the comment space. – Joander Vieira Cândido Jan 11 '16 at 10:40
  • I hear ya. You appear to have a nice answer above. I was just alerting you to the reality of how people get pinged – Drew Jan 11 '16 at 14:56
  • It is very slow – fatherazrael Oct 31 '19 at 11:39
5

The LDAP client is setting a "client-requested" size limit of 2000. This client-requested limit cannot override the limits set in the configuration of the server. No matter what the client-requested size limit is, the server's size limit overrides it. Contact your directory server administrator and ask that the size limit be increased.

Terry Gardner
  • 10,957
  • 2
  • 28
  • 38
  • if you were not able to change it in the request the "Active Directory Users and Computers" will also wont work .when i get the message on the 2000 entries limit on that tool i change the settings to a higher value and it works. so what you wrote does not sound correctly to me. – Asaf Magen Jun 24 '21 at 14:44