There's a problem: we have approx 48,000 accounts, but the LDAP server (Active Directory) is configured to limit all searches to a maximum of 1,000. Since auth_ldap_sync_users.php seems to rely on grabbing all accounts from a single query, this looks like the script may after all be of no use to us.
Any suggestions? I'd be very grateful to hear from anyone who has thousands of LDAP accounts and how they sync them. I'm trying to imagine a rewrite of the script to query users in smaller batches but I'm not sure how that could best be done.
1.- Increase AD query limit.
2.- Patch PHP LDAP extension to use Paged Results.
Have a look at this thread http://moodle.org/mod/forum/discuss.php?d=28791
The PHP developers have told me they won't add Paged Results support in 4.x or 5.0.x, as they only add new features to the latest stable version. They are testing the patch right now and might get added in the next 5.1.x release.
I have had a similar problem (1000 items returned) and I was able to ask our network manager to increase the MaxPageSize value from 1000. See entry from MS KB article below.
MaxPageSize - This value controls the maximum number of objects that are returned in a single search result, independent of how large each returned object is. To perform a search where the result might exceed this number of objects, the client must specify the paged search control. This is to group the returned results in groups that are no larger than the MaxPageSize value. To summarize, MaxPageSize controls the number of objects that are returned in a single search result.
Default value: 1,000
Unfortunately this did not appear to change the number of entries returned and so my current (temporary) solution is to add sufficient of the sub contexts into my search that I get all of our users.
Did you get this working in the end ?
As suggested in this post http://moodle.org/mod/forum/discuss.php?d=88626, our solution consists of
- a Perl script connects to AD LDAP and grabs all the records
- a hack to function sync_users() in auth/auth.php to invoke the Perl script and return the results in an array
- the contents of the array are used to populate the temp table created in auth/auth.php
I'm attaching the perl script and the code fragment in auth/auth.php that calls it (we're using a heavily hacked auth/auth.php, so attaching the whole file won't make a lot of sense, but hopefully, it shouldn't be too difficult to make sense of the attached code snippet).
Maybe in future we'll package the solution as a separate module, or incorporate it as a configurable option in the current module.
Hope this helps someone.
The PHP code fragment replaces the part of auth.php between:
// prepare some data we'll need (around line 607 in our copy of aiuth.php)
/// preserve our user database (around line 657 of auth.php)
Hope that helps.
PS: Apologies for the late response
thanks a lot for posting that. I recently needed to do this for Moodle 2.1 and Moodle 2.2.
The changes necessary to have this work on Moodle 2.1 / 2.2 can be found here: https://github.com/brki/moodle/commit/adde0b21f2bc14dee7f7775911ecda89250ad0e6
And an explanation of the what and why and how to use it can be found here: https://github.com/brki/moodle/blob/mdl22-ldap-perl-paging/README.txt
I've implemented your code for Moodle 2.2. When I turn the LDAP sync command, I get an the message below:
Connecting to LDAP server...
Creating temporary table tmp_extuser
Getting users from ldap for context: 'ou=Adjunct Faculty,ou=Accounts,dc=xxx,dc=xxx,dc=edu' ...
Finished getting users from ldap for context 'ou=Adjunct Faculty,ou=Accounts,dc=xxx,dc=xxx=edu'.
auth_ldap_connecterrorExiting sync script; something went wrong while trying to get a list of users
Can you give me any idea what I should check for that could have gone wrong? I've tried turning on all sorts of debugging but can't get anything more specific. I have regular LDAP auth and LDAP sync working fine, so I know the issue has to be related to implementing the paging functionality.
Any suggestions would help.
I was able to find the issue causing my ldap_connect error. It was because I had both a primary and a failover server specified in my Host URL. Once I removed the failover server IP, the sync ran perfectly.
Does anyone know what would need to be modified about the paging code to allow for a failover server to be present in the Host URL settings?
Thank you very much for posting this. It worked like a charm in our installation.