Sync Users from MS AD to Moodle

Sync Users from MS AD to Moodle

by Mina Attia -
Number of replies: 17

i have performed LDAP authentication with moodle and it was successfully established and now MS AD users can authenticate moodle with their AD identities
The issue is i have enabled the LDAP users sync job task schedule to sync all AD users to moodle but unfortunately no users are synced to AD and the task last run is status is Never and the task next run is ASAP.

and i have tried to run 
php moodle/admin/tool/task/cli/schedule_task.php --execute\\auth_ldap\\task\\sync_task

and the result was 

PHP Warning:  require(/root/moodle/admin/tool/task/cli/../../../../config.php): failed to open stream: No such file or directory in /root/moodle/admin/tool/task/cli/schedule_task.php on line 27

PHP Fatal error:  require(): Failed opening required '/root/moodle/admin/tool/task/cli/../../../../config.php' (include_path='.:/usr/share/php') in /root/moodle/admin/tool/task/cli/schedule_task.php on line 27



please advise 


Attachment mina.JPG
Average of ratings: -
In reply to Mina Attia

Re: Sync Users from MS AD to Moodle

by Ken Task -
Picture of Particularly helpful Moodlers

cli tools will look to relatively addressed location of the config.php file which contains variables (like DB user, DB host, etc) they need and to find other directories of Moodle code.

Thus your ... PHP Warning:  require(/root/moodle/admin/tool/task/cli/../../../../config.php)

Suggest running any cli php script from code root of moodle ...

cd /path/to/moodlecode/

php admin/tool/task/cli/schedule_task.php --execute="\\auth_ldap\\task\\sync_task"

then config.php can be found.

Also the sample command shown for schedule_task.php shows '--execute=' ... note the '=' and because \\ might be recognized as 'escapes' to whatever shell environment you might be in ... suggest putting quotes (") or ticks (') enclosing the task to run. ... use single \ not double.

Running from moodlecode/admin/tool/task/cli/:

php schedule_task.php --execute="\core\task\session_cleanup_task"

works in bash shell.

'SoS', Kene

In reply to Ken Task

Re: Sync Users from MS AD to Moodle

by Mina Attia -
Hello Ken
Thanks for your replay
i did what you advised me to do and i got this result

root@moodlefe:~/moodle/admin/tool/task/cli# pwd
/root/moodle/admin/tool/task/cli


root@moodlefe:~/moodle/admin/tool/task/cli# php admin/tool/task/cli/schedule_task.php --execute="\\auth_ldap\\task\\sync_task"
Could not open input file: admin/tool/task/cli/schedule_task.php


root@moodlefe:~/moodle/admin/tool/task/cli# php schedule_task.php --execute="\core\task\session_cleanup_task"
-su: php: command not found


root@moodlefe:~/moodle/admin/tool/task/cli# sudo php schedule_task.php --execute="\core\task\session_cleanup_task"
PHP Warning: require(/root/moodle/admin/tool/task/cli/../../../../config.php): failed to open stream: No such file or directory in /root/moodle/admin/tool/task/cli/sch edule_task.php on line 27
PHP Fatal error: require(): Failed opening required '/root/moodle/admin/tool/task/cli/../../../../config.php' (include_path='.:/usr/share/php') in /root/moodle/admin/t ool/task/cli/schedule_task.php on line 27

Thanks
In reply to Mina Attia

Re: Sync Users from MS AD to Moodle

by Ken Task -
Picture of Particularly helpful Moodlers

Is this accurate:

You show:

root@moodlefe:~/moodle/admin/tool/task/cli# pwd
/root/moodle/admin/tool/task/cli

pwd shows you are in /root/ ... and your moodle code is in there!!!???!!!

That's is a very strange place to have web accessible code on any Linux server.

This error you show:

PHP Warning: require(/root/moodle/admin/tool/task/cli/../../../../config.php): failed to open stream:

means the command issued cannot find the config.php file which is required to run any cli moodle code.

Any code on a web server that is to be accessed from the globe via browser is normally in a directory designated in config of whatever web service you are running.   Typical CentOS on Linux the config's for apache are in /etc/httpd/ and the main config file for httpd (the web service) has DocumentRoot defined as /var/www/html/

Your system????

So you've discovered, I think, some bigger issue with system ... and beyond the scope of these forums.

'SoS', Ken

Average of ratings: Useful (1)
In reply to Ken Task

Re: Sync Users from MS AD to Moodle

by Mina Attia -
Hello Ken

i found that i was running script from a wrong path and when i run it from the right path "/var/www/moodle/admin/tool/task/cli" i got the following result

Connecting to LDAP server...
Creating temporary table tmp_extuser
.....................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................Got 2405 records from LDAP
User entries to be updated: 9

and

User entries to be added: 2396

and at the end

... used 2905 dbqueries
... used 1.9697909355164 seconds
Scheduled task failed: LDAP users sync job (auth_ldap\task\sync_task),The given username contains invalid characters
Potential coding error - existing temptables found when disposing database. Must be dropped!

and still no users synced from AD to Moodle users

Thanks
In reply to Mina Attia

Re: Sync Users from MS AD to Moodle

by Ken Task -
Picture of Particularly helpful Moodlers

The given username contains invalid characters

Think there is a setting in LDAP config to accept just about any characters.

Check that and try it again.

'SoS', Ken

In reply to Ken Task

Re: Sync Users from MS AD to Moodle

by Ken Task -
Picture of Particularly helpful Moodlers

Just thought of something ...

Do you have a copy of moodle code in /root from initial install of moodle?

Is there a config.php file in /root/moodle/ ?

If no config.php file located there, the active/running under your web service moodle code is located else where on your system.

Check config of web server.

Might confuse, but one could try to locate config.php with following command as root user:

find / -name config.php

'SoS', Ken

Average of ratings: Useful (1)
In reply to Ken Task

Re: Sync Users from MS AD to Moodle

by Mina Attia -
Hello Ken

I run find / -name config.php and the result was as follow:-

root@moodlefe:/var/www/moodle/admin/tool/task/cli# find / -name config.php
/root/moodle/cache/classes/config.php
/root/moodle/mod/chat/gui_ajax/theme/course_theme/config.php
/root/moodle/mod/chat/gui_ajax/theme/bubble/config.php
/root/moodle/mod/chat/gui_ajax/theme/compact/config.php
/root/moodle/lib/editor/tinymce/plugins/spellchecker/config.php
/root/moodle/theme/boost/config.php
/root/moodle/theme/classic/config.php
/var/www/moodle/lib/editor/tinymce/plugins/spellchecker/config.php
/var/www/moodle/theme/squared/config.php
/var/www/moodle/theme/classic/config.php
/var/www/moodle/theme/boost/config.php
/var/www/moodle/theme/klass/config.php
/var/www/moodle/theme/eguru/config.php
/var/www/moodle/theme/contemporary/config.php
/var/www/moodle/theme/academi/config.php
/var/www/moodle/mod/bigbluebuttonbn/classes/locallib/config.php
/var/www/moodle/mod/chat/gui_ajax/theme/course_theme/config.php
/var/www/moodle/mod/chat/gui_ajax/theme/compact/config.php
/var/www/moodle/mod/chat/gui_ajax/theme/bubble/config.php
/var/www/moodle/config.php
/var/www/moodle/cache/classes/config.php
/var/www/moodledata/muc/config.php


Thanks
In reply to Mina Attia

Re: Sync Users from MS AD to Moodle

by Ken Task -
Picture of Particularly helpful Moodlers

Looks like active codee is in /var/www/moodle/

according to finding /var/www/moodle/config.php

So run cli scripts from /var/www/moodle/admin/cli/

cd /var/www/moodle/admin/cli/

run whatever in there ...

or cd /var/www/moodle

then give path to whatever script ...

php admin/cli/whatever.php


'SoS', Ken

In reply to Ken Task

Re: Sync Users from MS AD to Moodle

by Mina Attia -
Hello Ken
i have enabled debugging and i found the following

++ This page should be using theme contemporary which cannot be initialised. Falling back to the site theme eguru ++
* line 704 of /lib/outputlib.php: call to debugging()
* line 7348 of /lib/moodlelib.php: call to theme_config::load()
* line 702 of /lib/classes/user.php: call to get_list_of_themes()
* line 863 of /lib/classes/user.php: call to core_user::fill_properties_cache()
* line 832 of /lib/classes/user.php: call to core_user::get_property_type()
* line 59 of /user/lib.php: call to core_user::clean_field()
* line 924 of /auth/ldap/auth.php: call to user_create_user()
* line 50 of /auth/ldap/classes/task/sync_task.php: call to auth_plugin_ldap->sync_users()
* line 157 of /admin/tool/task/cli/schedule_task.php: call to auth_ldap\task\sync_task->execute()

also still find the error

Scheduled task failed: LDAP users sync job (auth_ldap\task\sync_task),The given username contains invalid characters
Potential coding error - existing temptables found when disposing database. Must be dropped!
In reply to Mina Attia

Re: Sync Users from MS AD to Moodle

by Ken Task -
Picture of Particularly helpful Moodlers

Well, we're slowly gaining on it ...

Version of Moodle is?

That is important considering this:

++ This page should be using theme contemporary which cannot be initialised. Falling back to the site theme eguru ++

That theme is compat up to 3.4.

https://moodle.org/plugins/pluginversions.php?plugin=theme_contemporary

The theme Moodle is trying to 'fall back to' is eguru.

eguru has only one version for 3.7.

https://moodle.org/plugins/pluginversions.php?plugin=theme_eguru

So suggest un-installing contemporary theme.

And if your moodle version is 3.7, set the theme to one that came with 3.7, that of 'boost' or 'classic'.

Believe it or not an issue with a theme does have some strange affects on other, seemingly not related, like LDAP sync.

So let's change the theme to a stock boot or classic to get around one error so that you can concentrate on the LDAP characters error.

And here's the official LDAP page on Moodle:

https://docs.moodle.org/37/en/LDAP_authentication

please review that and check your settings ... focus on anything you can find on characters.

Maybe Emma will drop by ... she is one of the recognized experts in community on LDAP!

Emma?  Where are you? ;)

'SoS', Ken




In reply to Ken Task

Re: Sync Users from MS AD to Moodle

by Mina Attia -
Hello Ken

i have changed the them and the same issue still exists

++ This page should be using theme contemporary which cannot be initialised. Falling back to the site theme boost ++
* line 704 of /lib/outputlib.php: call to debugging()
* line 7348 of /lib/moodlelib.php: call to theme_config::load()
* line 702 of /lib/classes/user.php: call to get_list_of_themes()
* line 863 of /lib/classes/user.php: call to core_user::fill_properties_cache()
* line 832 of /lib/classes/user.php: call to core_user::get_property_type()
* line 59 of /user/lib.php: call to core_user::clean_field()
* line 924 of /auth/ldap/auth.php: call to user_create_user()
* line 50 of /auth/ldap/classes/task/sync_task.php: call to auth_plugin_ldap->sync_users()
* line 157 of /admin/tool/task/cli/schedule_task.php: call to auth_ldap\task\sync_task->execute()

moodle version is
you are running 3.7+ (Build: 20190628)
In reply to Mina Attia

Re: Sync Users from MS AD to Moodle

by Ken Task -
Picture of Particularly helpful Moodlers

Uhhh ...

++ This page should be using theme contemporary which cannot be initialised. Falling back to the site theme boost ++

says whatever you did to change the theme, didn't take.

So one more item ... in config.php file of the active site, add the following line:

'$CFG->theme='boost';

Check your moodlecode/theme directory via anything you have to browse files.   If you should see only boost, classic, and eugru directories.  Any other directories like contemporary ... remove.

If successfully removed, do a query of DB like so:

mysql> select * from mdl_config_plugins where name like '%theme%';
+------+---------------+----------+------------+
| id   | plugin        | name     | value      |
+------+---------------+----------+------------+
| 1891 | theme_boost   | themerev | 1568043843 |
| 1953 | theme_classic | themerev | 1568043873 |
+------+---------------+----------+------------+
2 rows in set (0.00 sec)

Notice there's only the two in that table.

Issue could also be that LDAP sending characters that the DB won't except,

In your config.php file, what does the line for 'dbcollation' say?

What that shows must match DB.   3.7 now should be running not only Innodb but character set utf8mb4 with a collation of utf8mb4_something.

run these two scripts to see (and maybe fix):

from your moodle/admin/cli directory:

php mysql_compressed_rows.php -i

You should see:

Database engine:       InnoDB
innodb_file_per_table: ON
innodb_file_format:    Barracuda

Run

php mysql_collation.php --list

at the tail end of output, you should see something like:

mdl_workshopform_rubric_config          utf8mb4_general_ci
    layout                              utf8mb4_general_ci
mdl_workshopform_rubric_levels          utf8mb4_general_ci
    definition                          utf8mb4_general_ci

Table collations summary for https://yourserver:
utf8mb4_general_ci: 1494

which says all tables (1494 on the site I am looking at) have utf8mb4_general_ci collations.

That collation must match what's in conflg.php.

If you need to fix,

php mysql_collation.php

shows how to fix.  Read the screen and type command with options to change.

Now ... what language character set is your LDAP running?   Dunno how to find that out cause I don't run LDAP.

In the set up of LDAP in your moodle:

/admin/settings.php?section=authsettingldap

It does have a setting for LDAP encoding - which, by default, is set to utf-8.

and comment under that says:

"Encoding used by the LDAP server, most likely utf-8. If LDAP v2 is selected, Active Directory uses its configured encoding, such as cp1252 or cp1250."

Your LDAP should be version 3.

Have no idea how many users need to be sync'd, but one of them is using a character in the data being pulled into Moodle, that Moodle doesn't like.

That's all I can suggest.

Emma ... handing this one over to you! smile

'SoS', Ken

Average of ratings: Useful (1)
In reply to Ken Task

Re: Sync Users from MS AD to Moodle

by Emma Richardson -
Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Plugin developers
Sorry, guys. Just saw this. Your problem is that you have a username with a character that moodle does not accept. Ken already pointed this out. I often find an errant space or apostrophe causes this. There are two ways to fix this:
1. As Ken already mentioned, there is a setting in admin setting to allow special characters in usernames. Find the setting and enable it. Rerun the sync.
2. Find the username that contains the special character and change it.

Also noted, it appears that you do not have cron running. All your regular tasks are showing as never have run. You need to set up cron to run automatically. You can find instructions in the installation documents for your particular operating system.
Average of ratings: Useful (2)
In reply to Emma Richardson

Re: Sync Users from MS AD to Moodle

by Mina Attia -
Hello Emma
thanks for your replay
how can i allow special character in usernames ?
from where ?
Thanks
In reply to Mina Attia

Re: Sync Users from MS AD to Moodle

by Emma Richardson -
Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Plugin developers
Site security settings - just search extended characters
Average of ratings: Useful (1)
In reply to Emma Richardson

Re: Sync Users from MS AD to Moodle

by Mina Attia -
Hello Emma

Can you please guide me to the path of the file that i need to modify ?
and what is the parameter i need to change

Thanks
In reply to Mina Attia

Re: Sync Users from MS AD to Moodle

by Mina Attia -
Dear All

It is working now after allowing extended characters

Many Thanks Ken and Emma
really i appreciate your help and support