Moodle 2.5 - Google Apps User Synch fails to create new accounts

Moodle 2.5 - Google Apps User Synch fails to create new accounts

by David Morrow -
Number of replies: 34

Greetings!

The short question:

Are there any obvious reasons why upgrading from 2.4 to 2.5 would break a functioning Google Apps User Sync - and are there any easy solutions?

I'm keeping this question general for now, but will be glad to post any further information that may help anyone out there help me.

The situation, in general, is that we had set up Google Apps single-sign-on from our Moodle, using the Moodle in New Zealand Schools files - including SAML authentication, the gaccess block (containing the links to google docs and calendar), and the Google User Sync block (to create new Google accounts). All was working well until we upgraded to 2.5. The single-sign-on from Moodle to Google Apps still works for existing Google Apps accounts, but the user sync no longer creates new accounts. I have replaced all of the components with the latest ones, as far as I can tell.

Could Moodle's new password salting procedure have anything to do with it?

Thanks in advance!

David

Average of ratings: -
In reply to David Morrow

Re: Moodle 2.5 - Google Apps User Synch fails to create new accounts

by Rob Johnson -

Hi David,

I had a similar problem using 2.1.  I added the code in this thread to get user creation working.  I do not run the google cron to create users, and that code adds an event handler for the task.

If you delete a user in Moodle, does the user get deleted in Google?

Average of ratings: -
In reply to Rob Johnson

Re: Moodle 2.5 - Google Apps User Synch fails to create new accounts

by David Morrow -

Rob - Thanks for the rapid reply!

I just deleted a Moodle user, ran the gdata cron, and he was not deleted from Google. Even though the Google User Sync block status is "Authentication with Google Apps was successful".

You mention some code, but I didn't get a link or anything. Do you mind reposting it?

Thanks, and I welcome any further insights you have!

David

Average of ratings: -
In reply to David Morrow

Re: Moodle 2.5 - Google Apps User Synch fails to create new accounts

by Rob Johnson -

In the sync settings, you will see a checkbox for "Enable events."  That is how I deal with user sync.  I do not run the gdata cron.  It has deleted users on me that were not part of the sync.  Do you have that box checked?

The code I mentioned, but forgot the link is here.  It adds an event handler to create users in Google when one is created in Moodle.  Again, this happens without running the gdata cron.

Average of ratings: -
In reply to Rob Johnson

Re: Moodle 2.5 - Google Apps User Synch fails to create new accounts

by David Morrow -

I had not checked that box. I was under the impression that that setting only applies to Google accounts that are already in place. I will try it and report back. The same for that code you referred to. I thought I had read everything about Google Apps in Moodle.org, but don't remember seeing this. I may have skipped it because we are not enabling GMail (only Drive and Calendar), and at first glance it appears to pertain only to GMail. But, if I understand it correctly now, it would also work in our situation. I'll try it and report back.

Thanks,

David

Average of ratings: -
In reply to Rob Johnson

Re: Moodle 2.5 - Google Apps User Synch fails to create new accounts

by David Morrow -

Sorry  am so late with this update.

I did enable events in the sync block settings and added the two pieces of code you referred to.

But no change.

I did notice that our config.php file looks like it may be the original one from our 1.8 or 1.9 installation. I found this when researching why another plugin was not working correctly.

Is there anything in the config.php file - or that if missing from the config.php file - could cause a problem with Google User Sync block?

Thanks!

Average of ratings: -
In reply to David Morrow

Re: Moodle 2.5 - Google Apps User Synch fails to create new accounts

by Rob Johnson -

I do not have an answer about the config file, but mine is the same as I used with 1.9.  I re-read your original post.  What changed in the password salt with 2.5?

I have recently upgraded to the latest 2.4, and I am having problems with accounts not getting deleted in Google when deleted in Moodle.  I wish one of the developers of the New Zealand schools package would chime in.

Average of ratings: -
In reply to Rob Johnson

Re: Moodle 2.5 - Google Apps User Synch fails to create new accounts

by David Morrow -

Rob,

Thanks for the reply!

I'll say this like I know what I'm talking about: In 2.5, Moodle went from a site-wide password salt to a per-user salt (http://docs.moodle.org/25/en/Password_salting).

The main reason I thought that that might have something to do with my problem is that in the database table for the gapps block (again, I'm kinda just stringing words together - I am a former science teacher, not a programmer!), the password field for all of the users whose accounts failed to be created  is really different from all of the ones created before the update to 2.5. Before 2.5, all passwords are 30 alpha-numeric characters. After 2.5, they are all 60 characters and include symbols - and they all start with "$2y$10$". I did increase the password field size in this table for 30 to 60, based on someone else's suggestion.

I don't know enough to say that any of this is related. But I think I've checked everything that I can. The sync block even reports that authentication with Google Apps is successful. Our single-sign-on works with accounts that were created before the upgrade and with Google accounts that I create manually.

Our students passwords are less than 8 characters long, but that didn't seem to matter before the 2.5 upgrade. If I understand it correctly, it shouldn't matter now - Moodle is the authentication agent. But I might not understand it correctly. I look at it as Google saying, "Any friend of Moodle is a friend of mine - come on in!". Plus, even our teachers who have 8-character passwords are not getting accounts created.

The real techs are upgrading us from 2.5 to 2.5.1 as I type this - we'll see if that helps!

Thanks again - and to anyone else who can offer some help.

Average of ratings: -
In reply to David Morrow

Re: Moodle 2.5 - Google Apps User Synch fails to create new accounts

by David Morrow -

Updating to 2.5.1 went well - the real techs handled that!

No change in the situation with the Google User Sync block not creating new users.

I did turn debugging on and got the following when loading the Add Users to Sync page:

Strict Standards: call_user_func() expects parameter 1 to be a valid callback, non-static method block_gdata::view() should not be called statically in /usr/share/moodle/lib/blocklib.php on line 1606

Strict Standards: Non-static method block_gdata::has_capability() should not be called statically in /usr/share/moodle/blocks/gdata/block_gdata.php on line 252

I am not a programer, but am pretty good at following instructions, in case anyone has any suggestions.

Thanks!

Average of ratings: -
In reply to David Morrow

Re: Moodle 2.5 - Google Apps User Synch fails to create new accounts

by David Morrow -

An addition, if it helps with diagnosis:

It appears that the sync is not totally "broken". If I filter the Add Users to Sync to only include those whose last login was before the update to Moodle 2.5 and then run the GData cron, it does sync some users.

Does this mean anything?

Thanks,

David

Average of ratings: -
In reply to David Morrow

Re: Moodle 2.5 - Google Apps User Synch fails to create new accounts

by Dora K. -

Hello!

I just completed the moodle-google apps integration process and status turned green, which means it should be working, but when I try to add users to sync I get an "error writing to database" message.

I turned on debugging and got the following:

 

Debug info: Data too long for column 'password' at row 1
INSERT INTO mdl_block_gdata_gapps (userid,username,password,remove,lastsync,status) VALUES(?,?,?,?,?,?)
[array (
0 => '9',
1 => 'teacher',
2 => '$2y$10$NBIH6n/dqwlWuWT/wgxzM.eUBDrCI00jdwiy0MgbiePT/qPQlKkia',
3 => 0,
4 => 0,
5 => 'never',
)]
Error code: dmlwriteexception
Stack trace:
  • line 426 of /lib/dml/moodle_database.php: dml_write_exception thrown
  • line 1089 of /lib/dml/mysqli_native_moodle_database.php: call to moodle_database->query_end()
  • line 1131 of /lib/dml/mysqli_native_moodle_database.php: call to mysqli_native_moodle_database->insert_record_raw()
  • line 460 of /blocks/gdata/gapps.php: call to mysqli_native_moodle_database->insert_record()
  • line 454 of /blocks/gdata/block_gdata.php: call to blocks_gdata_gapps->moodle_create_user()
  • line 272 of /blocks/gdata/block_gdata.php: call to block_gdata->addusers_process()
  • line ? of unknownfile: call to block_gdata::view()
  • line 1606 of /lib/blocklib.php: call to call_user_func()
  • line 35 of /blocks/gdata/index.php: call to block_method_result()

 

Could anyone help? I don't understand what is going wrong. Any ideas on what I should do?

Average of ratings: -
In reply to Dora K.

Re: Moodle 2.5 - Google Apps User Synch fails to create new accounts

by Rob Johnson -

Hi Dora,

I don't have a solution for you, but David, the original poster in this thread, reported a similar error here.  My guess is the block_gdata_gapps is getting a string of data larger than it is set to take.   If you are comfortable making changes to the database, you can likely clear the error.

Rob

Average of ratings: -
In reply to Rob Johnson

Re: Moodle 2.5 - Google Apps User Synch fails to create new accounts

by Dora K. -

Hi Rob!

Thanks for answering! I saw the other thread but nobody says how I could chaange the database. I searched a little bit around but couldn't find out where passwords are stored or how to change the length for that string. Could you please tell me how to do it? I would be grateful! 

Average of ratings: -
In reply to Dora K.

Re: Moodle 2.5 - Google Apps User Synch fails to create new accounts

by Rob Johnson -

Hi Dora,

I have never made this change myself, but I will explain what I think needs to be done.  I hope other Moodlers can add to this thread so you don't do anything to break your site.  Proceed at your own risk, and back up the block_gdata_gapps table before you do anything.

The password field in the gdata_gapps table is set to VARCHAR 32. 

 Gdata table

This sets the maximum number of bytes each entry in that column can hold.  I think you have a user where the password exceeds that limit.  Increasing the value using a tool like phpmyadmin beyond the long password should clear the error.

Again, I hope other Moodlers reply here to confirm my thinking.

Rob

Average of ratings: -
In reply to Rob Johnson

Re: Moodle 2.5 - Google Apps User Synch fails to create new accounts

by Dora K. -

Hello again! 

Thanks for helping me out!

I changed the password field in the database to 255 and managed to add a user to sync. When I run the blocks/gdata/cron.php I got the following:

 

Starting Moodle to Google Apps synchronization Error(1402): Error 1402: InvalidPassword Invalid Input: "$2y$04$uKR6RmY0bzf0RcJqWxRrXu7koCfPaD8yDVLS9L.Xxhj13m1Wv3xJq" Google Apps error: Google Apps error: The serverencountered the following errors processing the request: Error 1402: InvalidPassword Invalid Input: "$2y$04$uKR6RmY0bzf0RcJqWxRrXu7koCfPaD8yDVLS9L.Xxhj13m1Wv3xJq"Debugging Traceback (to hide, turn off debugging):line 225 of blocks/gdata/gapps.php: call to blocks_gdata_gapps->gapps_create_user()line 641 of blocks/gdata/gapps.php: call to blocks_gdata_gapps->create_user()line 52 of blocks/gdata/rest.php: call to blocks_gdata_gapps->sync_moodle_user_to_gapps()Debugging Traceback (to hide, turn off debugging):line 641 of blocks/gdata/gapps.php: call to blocks_gdata_gapps->create_user()line 52 of blocks/gdata/rest.php: call to blocks_gdata_gapps->sync_moodle_user_to_gapps() Number of Google Apps accounts deleted: 0 Number of Google Apps accounts created: 0 Number of Google Apps accounts updated: 0 Number of errors: 1 End Moodle to Google Apps synchronization Execution took 2.703872 seconds

Average of ratings: -
In reply to Dora K.

Re: Moodle 2.5 - Google Apps User Synch fails to create new accounts

by Rob Johnson -

I do not use the gdata cron.  For user sync, I have events enabled in the sync block, and added an event handler to create users.  The code for doing this is here.

Average of ratings: -
In reply to Rob Johnson

Re: Moodle 2.5 - Google Apps User Synch fails to create new accounts

by Dora K. -

I added the code you suggested but I didn't see any change. I added a user to be synced but it stays there forever. Status says "Failed to create Google Apps account". I must be doing something wrong but I can't figure out what it may be. Everything seems to be in place. I am getting kinda desperate! Any other ideas?

Average of ratings: -
In reply to Dora K.

Re: Moodle 2.5 - Google Apps User Synch fails to create new accounts

by Rob Johnson -

Do you have events enabled in the sync block?  There is a checkbox in the settings for it.

Does your single sign on work properly?  The Gmail block?   Is it just the user sync that is misbehaving?

Average of ratings: -
In reply to Rob Johnson

Re: Moodle 2.5 - Google Apps User Synch fails to create new accounts

by Dora K. -

Yes, I do have events enabled. 

Single sign on works properly for the users that were already created in google apps. 

The gmail block only works for one of the two admin accounts I have. For the rest of the users I get an error Sorry could not obtain mail.

 

Average of ratings: -
In reply to Dora K.

Re: Moodle 2.5 - Google Apps User Synch fails to create new accounts

by Rob Johnson -

For user sync, do you have the provisioning API enabled in the Google control panel.  Look in Domain settings > User settings.

The gmail block is puzzling.  Is the admin account that works the one that you use for the gsaml username/password?

Average of ratings: -
In reply to Rob Johnson

Re: Moodle 2.5 - Google Apps User Synch fails to create new accounts

by David Morrow -

Sorry I am just now getting back into the discussion - but I am afraid I won't be of much help.

Dora - it sounds like we are having the same problem. Especially if this is occurring in Moodle 2.5. Our Google Apps integration was working well in 2.4. I started a separate discussion (https://moodle.org/mod/forum/discuss.php?d=235389), asking if anyone has had success using Google User Sync block with 2.5. No one has replied, and I ma beginning to wonder if it is even possible.

I did increase the password field in the mdl_block_gapps table (to 512), and that did get rid of one particular error, but user creation did not resume.

I've tried it both with and without the code that Rob referred to. Still no user creation, but I did see that a user I deleted from Moodle was deleted from Google Apps - automatically.

Events and Provisioning API are enabled.

With debugging turned on, I do get the following on the "Add users to sync" and "Users being synced" pages of the Google User Sync block:

Strict Standards: call_user_func() expects parameter 1 to be a valid callback, non-static method block_gdata::view() should not be called statically in /usr/share/moodle/lib/blocklib.php on line 1606

Strict Standards: Non-static method block_gdata::has_capability() should not be called statically in /usr/share/moodle/blocks/gdata/block_gdata.php on line 252

I don't know enough to know if these or related or how to fix them. I do have access - line 1606 in /moodle/lib/blocklib.php is:

    return call_user_func(array('block_'.$blockname, $method), $param);

And /moodle/blocks/gdata/block_gdata.php on line 252 is:

        self::has_capability(true);

We still have the green check and the message that "Authentication with Google Apps was successful". And when I run the gdata cron, it appears to be synching the existing users.

Our Single Sign On still works great - but only for Google Apps users that were created before the update to 2.5.

It is only the new user creation that is broken.

I do notice that all of the Moodle accounts that do not get created in Google Apps have a longer password in the mdl_block_gapps table - and they all start with "$2y$10$". I understand that this is due to the new password salting system. I still wonder if that feature of 2.5 is what broke our user creation sync.

And, since so many other issues I've had were ultimately resolved with the appropriate check box, I'm hoping that this issue will be an easy fix! I just don't yet know what that fix is.

Thanks, Rob, for all of your help. Good luck Dora!

David

Average of ratings: -
In reply to David Morrow

Re: Moodle 2.5 - Google Apps User Synch fails to create new accounts

by Dora K. -
Hello again, Still haven't figured anything out. Provisioning API is enabled. The account that sees mails in gmail block is that of an administrator in both GApps and Moodle but not the one I used i the gsaml... :S
Average of ratings: -
In reply to Dora K.

Re: Moodle 2.5 - Google Apps User Synch fails to create new accounts

by Rob Johnson -

If you delete a user does that user get deleted in Google?  Be sure it is a user added to sync.

What is different about the admin account where the email block works?  That is puzzling.

Average of ratings: -
In reply to Rob Johnson

Re: Moodle 2.5 - Google Apps User Synch fails to create new accounts

by Jamie Burgess -

Can anyone confirm that the user-sync can work in Moodle 2.5+?

Average of ratings: -
In reply to Jamie Burgess

Re: Moodle 2.5 - Google Apps User Synch fails to create new accounts

by Kimber Warden -

Yes, I've got it working in 2.5. It was a bear, though!

Average of ratings: -
In reply to Kimber Warden

Re: Moodle 2.5 - Google Apps User Synch fails to create new accounts

by Sam Norman -

Hi Kimber,

Can you please share how you have managed to get this working. It appears that there a few people having issues.

Thanks heaps,

 

Cheers,

 

Sam.

Average of ratings: -
In reply to David Morrow

Re: Moodle 2.5 - Google Apps User Synch fails to create new accounts

by Conn Warwicker -
Picture of Core developers Picture of Plugin developers

We had/have all sorts of issues with this as well. When we looked further into it, it appeared that the code simply wasn't there to make it all work, we had to re-write it ourselves.

Average of ratings: -
In reply to Conn Warwicker

Re: Moodle 2.5 - Google Apps User Synch fails to create new accounts

by Rob Johnson -

Would you mind posting your code here?  I would likely help several other moodlers.

Average of ratings: -
In reply to Rob Johnson

Re: Moodle 2.5 - Google Apps User Synch fails to create new accounts

by Mike Feist -

Well, its a shame that no one wanted to share their solutions to this issue... I don't know about anyone else but I'm a server administrator and not a web developer and I wasn't in a position to rewrite the PHP code myself.

Anyhow, I was running Moodle 2.4.2 with everything working fine and I had this problem after a Moodle 2.5.x upgrade with the GData block not provisioning users. I found a solution. Bear in mind the solution was more of a workaround but it is now working correctly for two Moodles running 2.5.3 and 2.5.4 using the Google User Sync block.

The problem stems from the changes to the password salting in Moodle 2.5.x. Moodle imposes this change if you are running PHP 5.3.7 or newer. The previous method was a site-wide MD5 password hash which Google supports and this gets upgraded to individual Blowfish hashes which Google don't support.

The solution was to downgrade PHP to 5.3.4 which causes Moodle to revert to the previous MD5 password salting method. Most of our users use AD logins so this wasn't a huge issue for us.

After the downgrade I had to rename the cache folder in Moodledata to cache.old to remove cached password entries and allow the AD logins to work again.

IMPORTANT: Any users that have "native" Moodle accounts may have been upgraded to have the Blowfish password hash which wont allow them to log in and they will require a password reset to work again. This also includes the main Admin account. I already had another AD-based admin user set up so I was able to log in with that and reset my Admin user account with ease.

You can also blank the password in the database using a "blank" MD5 hash if you have no other way to reset it. Info about that here: https://moodle.org/mod/forum/discuss.php?d=18103

After those changes the Google User Sync started working again.

Obviously, we all have different considerations to take into account but that workaround was fine in my case. Sorry this answer comes a bit late but I did spend a long time scouring the web for solutions. I did try the alternative SimpleSAML route but the documentation is so vague I couldn't get it working properly.

 

Hope that Helps.

 

Regards,

Mike

Average of ratings: -
In reply to Mike Feist

Re: Moodle 2.5 - Google Apps User Synch fails to create new accounts

by Stuart Smith -

We are currently developing our fresh install of Moodle 2.6, (our live version till now being 1.9) and are big users of Google Drive and Docs, so this thread has been really interesting so far. I've been looking for a simple way of allowing SSO and sync with Google, our drive being to make Moodle the main portal for accessing all our resources. Could anyone point me in the direction of the most up to date blocks mentioned in this thread? Hopefully with our systems manager and web developer working on this too we will be happy to share any resolutions we hit upon with you folks.


Stuart

Average of ratings: -
In reply to Stuart Smith

Re: Moodle 2.5 - Google Apps User Synch fails to create new accounts

by Nagi Saeed -

Hi Stuart,

You can find the latest blocks, authentication and zend files at the Moodle in New Zealand schools website. Can't wait to see what you guys can come up with.

Good luck,

Nagi

Average of ratings: -
In reply to Conn Warwicker

Re: Moodle 2.5 - Google Apps User Synch fails to create new accounts

by Conn Warwicker -
Picture of Core developers Picture of Plugin developers

Finally managed to come up with what I hope is a solution to this for us.

Basically what I've done is the following:

- Downloaded and installed GAM on our Linux server (https://code.google.com/p/google-apps-manager/wiki/GettingStarted)

- Connected it to our Google Apps domain

- Wrote a shell script to call the GAM create user script 

- Added in a method to the block_gmail class to execute my shell script, passing in the username, firstname and lastname of the authenticated user

- That then executes and creates or tries to create the account through GAM


It seems to be working so far. SO when a user logs in, instead of just seeing "cannot obtain mail", there is a "retry" button there as well now which I've added, and that refreshes the page and brings through their account, if GAM managed to create it.


I can go into more detail with code examples and whatnot if anyone wants.


Average of ratings: -
In reply to Conn Warwicker

Re: Moodle 2.5 - Google Apps User Synch fails to create new accounts

by Rob Johnson -

Please post both the shell script and the modified block_gmail code.  This would be very helpful.  I upgraded from 2.4 to 2.6 a few weeks ago, and have been running GAM manually to keep users updated.

Thank you in advance.

Rob

Average of ratings: -
In reply to Rob Johnson

Re: Moodle 2.5 - Google Apps User Synch fails to create new accounts

by Conn Warwicker -
Picture of Core developers Picture of Plugin developers

I installed gam into our moodledata directory, in a folder called "_gam".

I've attached the relevant files I use.


Due to our set up I've got it so it only runs if the username is a number, as our student usernames are all just idnumbers. It also removes anything other than letters, hyphens and whitespace from the first and lastname, for security purposes.

That stuff is easy to change though.


This is on Linux by the way. Would have to be slightly different for Windows.


Average of ratings: -
In reply to Conn Warwicker

Re: Moodle 2.5 - Google Apps User Synch fails to create new accounts

by Rob Johnson -

Thank you Conn.  My web server is a Linux box, and student usernames are numbers.  Your code changes look very promising.  I will post back if I run into problems.

Average of ratings: -