Authentication

 
 
Picture of Dan Stowell
LDAP sync users - what to do if LDAP search limits the # of records it'll return
 
Hi - I'm looking at the auth_ldap_sync_users.php script, to try and import all of our user accounts from LDAP into Moodle.

There's a problem: we have approx 48,000 accounts, but the LDAP server (Active Directory) is configured to limit all searches to a maximum of 1,000. Since auth_ldap_sync_users.php seems to rely on grabbing all accounts from a single query, this looks like the script may after all be of no use to us.

Any suggestions? I'd be very grateful to hear from anyone who has thousands of LDAP accounts and how they sync them. I'm trying to imagine a rewrite of the script to query users in smaller batches but I'm not sure how that could best be done.
 
Average of ratings: -
Picture of Iñaki Arenaza
Re: LDAP sync users - what to do if LDAP search limits the # of records it'll return
Group DevelopersGroup Documentation writersGroup Particularly helpful Moodlers
There are two things you can do:

1.- Increase AD query limit.
2.- Patch PHP LDAP extension to use Paged Results.

Have a look at this thread http://moodle.org/mod/forum/discuss.php?d=28791

The PHP developers have told me they won't add Paged Results support in 4.x or 5.0.x, as they only add new features to the latest stable version. They are testing the patch right now and might get added in the next 5.1.x release.

Saludos. Iñaki.

 
Average of ratings:Useful (1)
Picture of Dan Stowell
Re: LDAP sync users - what to do if LDAP search limits the # of records it'll return
 
Aha - thanks for the pointer to that interesting discussion. Good luck with getting the Paged Results patch accepted.
 
Average of ratings: -
Picture of Steve Power
Re: LDAP sync users - what to do if LDAP search limits the # of records it'll return
 

Dan

I have had a similar problem (1000 items returned) and I was able to ask our network manager to increase the MaxPageSize value from 1000. See entry from MS KB article below.

MaxPageSize - This value controls the maximum number of objects that are returned in a single search result, independent of how large each returned object is. To perform a search where the result might exceed this number of objects, the client must specify the paged search control. This is to group the returned results in groups that are no larger than the MaxPageSize value. To summarize, MaxPageSize controls the number of objects that are returned in a single search result.

Default value: 1,000

Unfortunately this did not appear to change the number of entries returned and so my current (temporary) solution is to add sufficient of the sub contexts into my search that I get all of our users.

Regards
Steve

 
Average of ratings: -
Me :)
Re: LDAP sync users - what to do if LDAP search limits the # of records it'll return
 
Hi Steve
Did you get this working in the end ?

Kev
 
Average of ratings: -
Picture of Olumuyiwa Taiwo
Re: LDAP sync users - what to do if LDAP search limits the # of records it'll return
 
I don't know if this is still an issue for some users - the last post in the thread was in August 2009 - but thought I'll post our solution, just in case it helps anyone.

As suggested in this post http://moodle.org/mod/forum/discuss.php?d=88626, our solution consists of
  • a Perl script connects to AD LDAP and grabs all the records
  • a hack to function sync_users() in auth/auth.php to invoke the Perl script and return the results in an array
  • the contents of the array are used to populate the temp table created in auth/auth.php
The solution requires that Perl and the Perl-LDAP libraries be installed. On Centos/RedHat/Fedora systems, that's a simple yum install perl-LDAP. Unfortunately I don't know how easy or difficult it would be to get them installed on other Unix or Linux flavours or on Windows boxes.

I'm attaching the perl script and the code fragment in auth/auth.php that calls it (we're using a heavily hacked auth/auth.php, so attaching the whole file won't make a lot of sense, but hopefully, it shouldn't be too difficult to make sense of the attached code snippet).

Maybe in future we'll package the solution as a separate module, or incorporate it as a configurable option in the current module.

Hope this helps someone.
Muyi


 
Average of ratings:Useful (1)
Picture of Shane Wilson
Re: LDAP sync users - what to do if LDAP search limits the # of records it'll return
 
I would like to try your code, could you give me an idea about where exactly it fits in (or replaces) the auth.php file?

Thanks!
 
Average of ratings: -
Picture of Olumuyiwa Taiwo
Re: LDAP sync users - what to do if LDAP search limits the # of records it'll return
 
Hi Shane,

The PHP code fragment replaces the part of auth.php between:

// prepare some data we'll need (around line 607 in our copy of aiuth.php)
and
/// preserve our user database (around line 657 of auth.php)

Hope that helps.

Muyi
PS: Apologies for the late response
 
Average of ratings:Useful (1)
Picture of Brian King
Re: LDAP sync users - what to do if LDAP search limits the # of records it'll return
Group Developers

Hi Muyi,

thanks a lot for posting that.  I recently needed to do this for Moodle 2.1 and Moodle 2.2.

The changes necessary to have this work on Moodle 2.1 / 2.2 can be found here: https://github.com/brki/moodle/commit/adde0b21f2bc14dee7f7775911ecda89250ad0e6

And an explanation of the what and why and how to use it can be found here: https://github.com/brki/moodle/blob/mdl22-ldap-perl-paging/README.txt

Cheers,

Brian

 
Average of ratings:Useful (1)
Picture of Janet Smith
Re: LDAP sync users - what to do if LDAP search limits the # of records it'll return
 

Hi Brian,

I've implemented your code for Moodle 2.2. When I turn the LDAP sync command, I get an the message below:

Connecting to LDAP server...
Creating temporary table tmp_extuser
Getting users from ldap for context: 'ou=Adjunct Faculty,ou=Accounts,dc=xxx,dc=xxx,dc=edu' ...
Finished getting users from ldap for context 'ou=Adjunct Faculty,ou=Accounts,dc=xxx,dc=xxx=edu'.
auth_ldap_connecterrorExiting sync script; something went wrong while trying to get a list of users

Can you give me any idea what I should check for that could have gone wrong? I've tried turning on all sorts of debugging but can't get anything more specific. I have regular LDAP auth and LDAP sync working fine, so I know the issue has to be related to implementing the paging functionality.

Any suggestions would help.


Thanks,
Janet

 
Average of ratings: -
Picture of Janet Smith
Re: LDAP sync users - what to do if LDAP search limits the # of records it'll return
 

I was able to find the issue causing my ldap_connect error. It was because I had both a primary and a failover server specified in my Host URL. Once I removed the failover server IP, the sync ran perfectly.


Does anyone know what would need to be modified about the paging code to allow for a failover server to be present in the Host URL settings?

 
Average of ratings: -
Picture of Guillermo Vargas
Re: LDAP sync users - what to do if LDAP search limits the # of records it'll return
 

Thank you very much for posting this. It worked like a charm in our installation.

Guillermo.

 
Average of ratings: -
Picture of priya virvani
About Moodle
 

Respected All

Can Anyone tell me how to i begin with moodle? I have no idea about it. I just download it and now what i do in that??

 
Average of ratings: -