7

I am working in C#, and am trying to use DirectorySearch to query the groups of an extremely large Microsoft ActiveDirectory LDAP server.

So, in my application, I'm going to have a paged list of groups, with searching capability. Naturally, I don't want to hammer my LDAP server with passing me the entire result set for these queries every time I hit "Next Page".

Is there a way, using DirectorySearch, to retrieve ONLY a single arbitrary page's results, rather than returning the entire result-set in one method call?

Similar questions:

Many questions like these exist, where someone asks about paging (meaning from LDAP server to app server), and gets responses involving PageSize and SizeLimit. However, those properties only affect paging between the C# server and the LDAP server, and in the end, the only relevant methods that DirectorySearch has are FindOne() and FindAll().

What I'm looking for is basically "FindPaged(pageSize, pageNumber)" (the pageNumber being the really important bit. I don't just want the first 1000 results, I want (for example) the 100'th set of 1000 results. The app can't wait for 100,000 records to be passed from the LDAP server to the app server, even if they are broken up into 1,000-record chunks.

I understand that DirectoryServices.Protocols has SearchRequest, which (I think?) allows you to use a "PageResultRequestControl", which looks like it has what I'm looking for (although it looks like the paging information comes in "cookies", which I'm not sure how I'd be supposed to retrieve). But if there's a way to do this without rewriting the entire thing to use Protocols instead, I'd rather not have to do so.

I just can't imagine there's no way to do this... Even SQL has Row_Number.

UPDATE: The PageResultRequestControl does not help - It's forward-only and sequential (You must call and get the first N results before you can get the "cookie" token necessary to make a call to get result N+1).

However, the cookie does appear to have some sort of reproducible ordering... On a result set I was working on, I iterated one by one through the results, and each time the cookie came out thusly:

1: {8, 0, 0, 0}
2: {11, 0, 0, 0}
3: {12, 0, 0, 0}
4: {16, 0, 0, 0}

When I iterated through two by two, I got the same numbers (11, 16). This makes me think that if I could figure out the code of how those numbers are generated, I could create a cookie ad-hoc, which would give me exactly the paging I'm looking for.

Community
  • 1
  • 1
ChristopherBass
  • 301
  • 4
  • 17
  • That first link should work for you. Just set `PageSize` to the appropriate value, use `System.Linq` extension methods on it, and use `Take` and `Skip` to implement your paging. According to the answer it comes back as one enumeration, but does appropriate paging under the covers. The only real trick is how to jump to a specific page/do paging on separate web requests (if you have a web app), since I don't know that there is a way to skip `n` pages of results without actually retrieving them. If you don't have this requirement, then you don't have that problem tho :) – Merlyn Morgan-Graham Oct 06 '11 at 23:38
  • If you have that Req, EJP's answer looks promising, tho... (`PageResultRequestControl`). – Merlyn Morgan-Graham Oct 06 '11 at 23:45
  • @Merlyn Morgan-Graham We're all talking about the same thing here. The control is how it is implemented under the hood: what form it takes in the various C# APIs is certainly not my area. – user207421 Oct 07 '11 at 02:31
  • @MerlynMorgan-Graham Yeah, it's for a web app. I guess I didn't make that clear above. – ChristopherBass Oct 10 '11 at 14:07

3 Answers3

3

The PageResultRequestControl is indeed the way to do this, it's part of the LDAP protocol. You'll just have to figure out what that implies for your code, sorry. There should be a way to use it from where you are, but, having said that, I'm working in Java and I've just had to write a dozen or so request controls and extended-operation classes for use with JNDI so you might be out of luck ... or you might have to do like I did. Warning, ASN.1 parsing follows not that far behind :-|

user207421
  • 305,947
  • 44
  • 307
  • 483
  • 1
    Looks like there might be a code same that uses that control from C# on this page: http://stackoverflow.com/questions/1646518/iplanet-ldap-and-c-pageresultrequestcontrol – Merlyn Morgan-Graham Oct 06 '11 at 23:43
  • 1
    So, testing this method while reading this: http://msdn.microsoft.com/en-us/library/bb332056.aspx#sdspintro_topic5_pagedsearch My understanding is, the PageResultRequestControl code will let you page through, as long as you maintain the Cookie byte[] from previous calls (whether by sending it with to the client, or cacheing or what have you). However, in order to request an arbitrary page, or even page backwards, you'd have to hand-craft the cookie, and I'm not sure how reasonable it is to do that. – ChristopherBass Oct 10 '11 at 14:53
  • @Janus: Yeah, that sample shows a while loop to advance to the requested page. Looks like this control won't be helpful... – Merlyn Morgan-Graham Oct 10 '11 at 18:58
3

Sadly, it appears there may not be a way to do this given current C# libraries.

All of the standard C#4.0 LDAP libraries return Top-N results (As in, FindAll(), which returns every result, FindOne(), which returns the first result, or SearchResult with PageResultRequestControl, which returns results N through N+M but requires you to retrieve results 1 through N-1 before you'll have a cookie token that you can pass with the request in order to get the next set.

I haven't been able to find any third-party LDAP libraries that allow this, either.

Unless a better solution is found, my path forward will be to modify the interface to instead display the top X results, with no client paging capabilities (obviously still using server-side paging as appropriate).

I may pursue a forward-only paging system at a later date, by passing the updated cookie to the client with the response, and passing it back with a click of a "More Results" type of button. It might be worth pursuing at a later date, whether or not these cookies can be hand-crafted.

UPDATE: I spoke with Microsoft Support and confirmed this - There is no way to do dynamic paging with LDAP servers. This is a limitation of LDAP servers themselves.

You can use Protocols and the Paging control (if your LDAP server supports it) to step forward at will, but there is no cross-server (or even cross-version) standard for the cookie, so you can't reasonably craft your own, and there's no guarantee that the cookie can be reused for repeated queries.

A full solution involves using Protocols (with Paging as above) to pull your pageable result set into SQL, whether into a temp table or a permanent storage table, and allow your user to page and sort through THAT result set in the traditional manner. Bear in mind your results won't be precisely up to date, but with some smart cache updating you can minimize that risk.

ChristopherBass
  • 301
  • 4
  • 17
0

Maybe you want to iterate through your "pages" using the range-attribute accordingly:

----copy & paste----

This sample retrieves entries 0-500, inclusively.

DirectoryEntry group = new DirectoryEntry("LDAP://CN=Sales,DC=Fabrikam,DC=COM");
DirectorySearcher groupMember = new DirectorySearcher
    (group,"(objectClass=*)",new string[]{"member;Range=0-500"},SearchScope.Base);
SearchResult result = groupMember.FindOne();
// Each entry contains a property name and the path (ADsPath).
// The following code returns the property name from the PropertyCollection. 
String propName=String.Empty;
foreach(string s in result.Properties.PropertyNames)
{
    if ( s.ToLower() != "adspath")
    {
      propName = s;
      break;
    }
}
foreach(string member in result.Properties[propName])
{
     Console.WriteLine(member);
}

----copy & paste----

for more Information see:

Enumerating Members in a Large Group https://msdn.microsoft.com/en-us/library/ms180907.aspx

Range Retrieval of Attribute Values https://msdn.microsoft.com/en-us/library/cc223242.aspx

Searching Using Range Retrieval https://msdn.microsoft.com/en-us/library/aa367017.aspx

Bernhard
  • 2,541
  • 1
  • 26
  • 24