Hello,
I just upgraded my Moodle installations to version 3.3. My users login using Oauth2 with their LinkedIn accounts. I could not however find a way how to migrate them to the new Oauth2 core plugin, therefore I am still using the old contributed one.
Is there any way to achieve the above goal? Thank you!
Were you using this one:
https://moodle.org/plugins/auth_googleoauth2
Think this will require direct manipulation of mdl_users table. If you look at that table you will see that two columns in that table are different ... the 'auth' column is set to 'googleoath2' and the user_name column has used 'social_user_#' as opposed to a 'ktask' or 'ken.task' or other. And the password field is set to 'not cached' right now.
One way to make such massive changes, used to be export all users to a csv file. Then manipulate that csv file for desired results (did this once when Moodle moving from all manual to LDAP).
You won't be creating new accounts but editing existing.
In 3.3 new table: mdl_oauth2_user_field_mapping exist (fresh install). The mdl_user table under auth now has oath2 and username is now an EMail address. Password is blank - that's because the password, like LDAP, not stored in Moodle.
The new table ... mdl_oauth2_user_field_mapping ... has a 'user_modified' column which appears to be the user ID number.
You might have to refer to:
https://github.com/rabser/moodle-auth_googleoauth2/issues
Suggest using one user for testing when using CSV updates like that.
The tricky part of this ... might involve turning off current oath2 authentication, turning on the new one (after it is configured) and testing with the one user you've changed via CSV.
Might be best to set up a clone of production where you can 'tinker' without fear! ;)
'spirit of sharing', Ken
Thank you very much!
Yes, this is the one I am still using. For the time being I will postpone the tinkering but I think I will give it a try (especially if no smoother way comes up). The good thing is that I can afford even a fresh re-install and user re-registration during the summer when the semester is over but I was wondering whether there is a cleverer approach.
'cleverer' sometimes gets one in trouble ... been there done that! (not very clever).
'Smoother' way might have to be provided by the maker of the plugin ... some conversion script of some sort. See from the git hub, however, it's changed ownership and am not sure I'd look for a freebie conversion script in the near future. Guess you could contact current maintainer and inquire.
Good that you have a break ... perfect time to take care of it. Fresh install will probably clean up things you've not discovered yet. ;)
'spirit of sharing', Ken
"Cleverer" was related to a potential alternative of a fresh install. I apologize if I did not make it clear enough and this possibly made it sound rude. Thank you very much again, and cheers! (And, summer is not so much far ahead, and you are right that cleaning old and forgotten stuff might bring some pleasant gains.)
No offense take about 'clever'! I know more about how not to do something than do it right the first time ... no matter how 'clever' I think I might be!!!! ;)
It has been a great plugin and one that drew enough attention to be partially responsible for Moodle HQ decision to move in that direction (it's time!).
And am glad to see the current maintainer dropped in here ... would be nice to have a script to convert accounts. But, then again, like I mentioned, a fresh install of a site, new users registering under the new authentication and new courses does also sound good.
I know of a Univ on West Coast of USA, that has a new Moodle every academic year ... archiving old sites to another domain and allowing students/teachers continued access to old courses ... but can't re-submit assignments, add to any forum, etc.. Am almost willing to bet they get the bugs out of new at the start and then pretty much 'surf' the remainder of that academic year.
Must be nice! I wouldn't know, however.
'spirit of sharing', Ken
I'm the new lead mantainer for the plugin.
As stated in the plugin page by the old mantainer (Jerome Mouneyrac) this plugin will not be ported to moodle 3.3 because it can cause conflict with the oauth2 embedded authentication.
My job will be mantaining the plugin only for 2.x / 3.1 / 3.2 releases, and in particular for Moodle 3.1 which is a LTS release.
Thanks to Ken for sharing some hints i'll take a look asap, and i could do a special branch for 3.3, but i would'nt overlap the moodle core functionalities, so i'm in doubt if it's correct publish an official release.
Comments and suggestions are welcome.
Thank you for joining the discussion! The plugin that you are taking care of is working so far perfectly under Moodle 3.3. I have disabled the core one to avoid potential conflicts. So if no bug issues are pending, guys like me can put up with some waiting and migrate as soon as possible to the core Oauth2.
Hi Sergio! Thanks for your message!
I've been using the "old auth2" plugin and I confess I'm not 100% confident about upgrading my moodle 3.2+ to 3.3.
I'm not a developer so I don't know if I'll be able to do this "manual-bit-byte-migration". I know it's not your responsability, but could you help me with this info I would be very thankful:
1) I have many users created using "linkedin", "facebook" and Google+". If I update my moodle, will they be migrated to this new "Auth2 core"?
2) If thre's no automatic migration, is there a script "how-to" migrate?
3) If it doen't work properly, can I unistall the "core auth" and keep using the old one?
I really appreciate your help.
Best Regards
Actually i could'nt answer to your question, as i didn't have time to take a look into the new code: hope that who developed the new core functionality could answer to your doubt: they know better than me their code.
My effort it's mostly oriented to guarantee a support for the releases older than 3.3 until they lives and are officially supported.
By now, if you are not urged to do this upgrade, my very personal suggestion is: wait some weeks...
Sergio, thanks for your message. I have the same feeling (to wait a while untill they get all the problems set).
In my opinion, moodle dev team has acted hastily releasing this new "core oauth2". Very fast, but inefficient.
I tried to install both oath2 on my DEV server and they didn't work properly. It means, I can't update my moodle anymore till this point is completely clarified by moodle team.
Again, thanks.
Best Regards,
Filipe
Sergio, you stated, "this plugin will not be ported to moodle 3.3 because it can cause conflict with the oauth2 embedded authentication." Can you amplify on that, please?
I have just upgraded from 3.1.7 to 3.3.1. My update script copied the auth/googleoauth2 plugin directory across to the the new install, and immediately after the upgrade, my site was still using the plugin. However, all attempts to get the core OAuth 2 functionality to work has failed - sessions time out after a short time (around a minute or two, often less). See my posts further down this thread.
Just about the only thing I haven't tried in my attempts to get the core OAuth 2 code to work is to uninstall the auth_googleoath2 plugin and delete its directory. I am very reluctant to try this for fear of not being able to re-enable it if that fails to solve the problem, so for the time being, I am assuming that if the auth_googleoauth2 plugin remains installed, but disabled, it should not cause any problems.
What do you think?
--- Les
Sorry for the late response, i was away for summer holidays.
Mainly i believe that is wrong duplicating inside a plugin the core functionality, so i believe that the core oauth2 should be the first choice, and our effort should be in helping the HQ to debug and correct the errors in the core code.
A plugin, even if disabled, can cause conflicts in loading libraries, so i suggest (my opinion...) that if the plugin is not used anymore, it should be removed from the filesystem.
Bye.
That's all great, Sergio, but:
At this point, I suspect the problem lies in the caching of tokens somewhere - maybe the server, maybe the browser, maybe Google and maybe all three. Once the current high level of activity on our site dies down, next week (I hope), then I'm going to do a 5 am test in which I log myself completely out of Google, switch to the core OAuth 2 module, disable the plugin, rename its subdirectory and purge all caches on the server, touch wood, say a prayer to the Blessed Saint Turing and try to log in using my Google account. If that works, trust me - I will be yelling it from the rooftops. ;)
--- Les
AFAIK there are two oauth2 plugins for moodle, the one who i'm currently mantaining and using is auth_googleoauth2: in this plugin there is no oauth2 token caching enabled nor needed for the authentication mechanism.
Please, post the errors that you're getting after switching and (re)moving the plugin, to better understand what it's happening.
Hi, Sergio,
I have a thread with a more comprehensive description of the problem at https://moodle.org/mod/forum/discuss.php?d=355492#p1434007. But what I said there is:
---------------------------
I went to Plugins -> Authentication -> Manage authentication,
disabled OAuth2 (googleoauth2) and enabled OAuth 2 (oauth2). Then I went
to Plugins overview, only to find there is no uninstall option for the googleoauth2 module.
So I moved auth/googleoauth2 out of the way by renaming it - after all, it was disabled. Refreshing the Plugins overview page got Moodle to complain that the plugin could not be found on the disk, followed by an option to update the database - so I did that, and thought I'd got googleoauth2 uninstalled.
I logged
out, then tried to log back in using the core OAuth2 Google button and was
taken to the Google account chooser, only to then get a BIG message that
googleoauth2 could not be found, and no way of getting the site to
load. (From memory, it was trying to invoke /auth/googleoauth2/google_redirect.php.)
---------------------------
I suspect that I'm going to have to:
in order to get this resolved. Then the question is, are my users going to have to go through a similar process of logging out and deleting cookies?
In any case, I can't try this until things go quiet on the server, some time early next week.
--- Les
Have you checked into the mdl_user table and editing via csv file that you manipulate?
User ID's wouldn't change then and the only thing that would change is the 'auth' column for how a user would be authenticated.
On the Google end, at least, the secret/key should be different as well as the call back URL - where one sets up the credentials in Google.
Had to do something similar for a site that had used manual/email for students originally and later that same academic year the entity had finally worked out their LDAP issues and wanted all students to authenticate via LDAP. Had to make a couple of runs via csv but eventually got all students auth vs. LDAP. Think you should be able to do the same.
'spirit of sharing', Ken
Hi, Ken,
"User ID's wouldn't change then and the only thing that would change is the 'auth' column for how a user would be authenticated."
I don't think the user ID changes anyway, although I could be wrong. The old plugin created user ID's like "social_user_n" and I don't think that switching to the new core OAuth2 functionality will change that - in fact, I just checked and on my test account, which I used to log in with both, it hasn't changed. It's the email address that is the identifier for OAuth 2, not the username.
"On the Google end, at least, the secret/key should be different as well as the call back URL - where one sets up the credentials in Google."
Agreed, and in fact, I deliberately created two distinct sets of secrets at Google for just that purpose. However, switching to the core OAuth2 still results in the original plugin callback URL being called after 90 seconds or so - I'm pretty sure that's why sessions were getting broken after a minute or two, and why I got big warnings when I renamed the googleoauth2 plugin directory. I checked that the callback URL was set correctly on the core OAuth2 setup, but Google was still using the old URL.
That's why I think this is a caching problem - so when things go quiet, early next week, I'm going to get up at 5 am, switch the config and purge every cached bit of session info and cookies I can think of. Hopefully, that will fix it and my users won't have too much trouble.
Thanks, Ken!
--- Les
In/on a Moodle 3.3 that cannot run the old plugin:
mysql> select id,auth,username,firstname,lastname,email from mdl_user where auth="oauth2";
+----+--------+------------------------+-----------+----------+------------------------+
| id | auth | username | firstname | lastname | email |
+----+--------+------------------------+-----------+----------+------------------------+
| 4 | oauth2 | mine@gmail.com | Ken | Task | mine@gmail.com
On a 3.2 that **does** use the old plugin:
mysql> select id,auth,username,firstname,lastname,email from mdl_user where auth like "google%";
+----+--------------+---------------+-----------+----------+----------------------+
| id | auth | username | firstname | lastname | email |
+----+--------------+---------------+-----------+----------+----------------------+
| 6 | googleoauth2 | social_user_1 | Ken | Task | mine@gmail.com
Moodle doesn't populate the password column.
which is why I think if one used csv from a spreadsheet that's been manipulated ...
googleoauth2 changed to oauth2
username changed from social_user_# to googleemailaddress
Might be the eaiest way to move all users to the new and keep them in courses in which enrolled, etc.
Google has always used EMail address for username, me thinks.
The user ID is what ties users to courses ... is it not?
'spirit of sharing', Ken
Hello Ken,
I solved it in a simple way.
I have set the authentication method again for: manual and username for email
This way when the user who was already registered click the authentication icon of google, it will be asked if you want to associate the existing account with google account.
If so, it will receive an email with the link that will simply click and the existing account (manual) will be associated with google account.
SQL to update user table
update mdl_user set username=email , auth='manual' where auth='googleoauth2'
'spirit of sharing'
To be useful, I need the error message exactly as it is printed out or,even better, enabling the moodle debugging to the max level the errors thrown in the error_log of the web server.
So i'll wait for your next tests...
OK, Sergio,
I was at my desk by 6:15 am this morning to attempt the changeover. I did all the things I suggested above:
I went through the whole two-factor process, only to see the URL bar redirect through /accounts/setSID to a near-blank screen with the message
"Authentication plugin googleoauth2 not found"
appearing twice. I even went to the Google API Manager and set all possible callbacks to the new core OAuth 2 module's address of "/admin/oauth2callback.php" but it still produces the same error message. If I've got my notes correct, the URL at that point is
which looks, to me, as though it's redirecting through the new OAuth 2 module.
With the googleoauth2 directory in place (but disabled) the error messages do not appear and the login appears to be successful - but within a minute or two, the session is broken and I am logged out (I'm testing by editing quiz question feedback - the symptom specifically is that draft edits cannot be saved and refreshing the site front page in another tab shows I'm logged ut).
I didn't up the level of logging, but the logs just show me logging in and out (I wasn't noting the specific times so can't correlate my actions against the logs) and selecting "Site errors" produces an empty report, so I'm not sure anything useful is being logged anyway.
And sure enough - no sooner had I reset the site to normal operation (which turned out to be tricky as it wasn't recognising the replaced auth/googleoauth2 directory at first) than a student logged in, before 7:30 am.
tl;dr - There is no "uninstall" option for the googleoauth2 plugin but just disabling it and moving it away is not enough to stop callbacks to it.
Happy to up the level of logging and correlate with the Apache logs if it will help. At this point, I'm flummoxed.
--- Les
Another curious thing: not only is there no "uninstall" link for this plugin, I can't find any "settings" link or admin page, either. Yet there must be one - the docs refer to it, and I must have set up the client ID and key or it wouldn't be working.
I've tried reinstalling the auth_googleoauth2 plugin - the installation just warned that the target directory already exists and will be removed - but reinstalling didn't seem to make any difference.
--- Les
I've no doubt that you're doing the things well, so these are my opinions:
Hope this helps.
Thanks, Sergio - I will give it another try tomorrow morning. I'll also up the level of logging.
--- Les
Apologies for the delayed reply - the site has been busy, as have I, and I've been traveling, to boot.
SUCCESS!
The missing link was this:
"Please note that the value "googleoauth2" it's written along all your users which where tied to it, so you need to alter your mdl_user records (DO A BACKUP FIRST!) to reset the authentication method for your users: change in the column auth from googleoauth2 to oauth2. Maybe start changing only your user and note that you can do the same by opening the moodle interface for you user and changing in the advanced settings of the user profile the authentication method and you should be ok."
As you suggested, I tried it with my own account first and after some testing, I breathed a sigh of relief and have now changed all the old googleoauth2 entries - for the SQL neophytes among us, the statement to do this is
UPDATE mdl_user SET auth = 'oauth2' WHERE auth = 'googleoauth2';
I'm hoping that the confirmation emails go out OK and don't get caught in spam traps, etc.
Thanks for your help with this - now it's done I can sleep a bit more easily. ;)
--- Les
Hi Les,
I've followed the same process and some of my users have received the email in their SPAM folder. So, my suggestion is to send a "newsletter" informing the users about the validation process and to check their spam folder just in case.
Anyway, As said in other entry of this threat it would be great if there was a chance to avoid this mail to the users that had been using the old OAuth module in previous Moodle versions. I assume it's not easy to fix from a tech point of view, bur from a user point of view it doesn't make any sense they've been login for years with their google account in the platform and now the have to validate it again.
Best regards,
Alejandro
Can anyone tell me why I'm getting the error message "The system account was not connected for offline access" when I go to enable LinkedIn and Facebook? I've successfully set up Oauth2 for Google and Microsoft but something isn't right for Facebook and LinkedIn. When I go to connect, it doesn't go to Facebook or LinkedIn for the final step (like it did for Google and Microsoft) and just returns me to the Moodle Oauth 2 service settings with that message displayed at the top. I've double checked all of the settings and they look correct. Purged Moodle cache, too, but I get the same result.
Thanks for any help.
Dave
Which release of moodle are you using ? did you configure the oauth2 plugin or you are using the native (Moodle >3.2) oauth2 feature ?
It's Moodle 3.4.1+ (Build: 20180208) Version 2017111301.04. I'm using the native Moodle oauth2 feature. I was able to get Microsoft and Google set up properly. Not happening with LinkedIn and Facebook. I double-checked all the settings as well as the endpoints and everything seems correct to me. I'm also logged into LinkedIn and Facebook developer accounts but when I try to connect to authorize, it just returns to the Moodle Oauth2 Service page instead of going to LinkedIn and Facebook to get approval. The message at the top says - "The system account was not connected for offline access".
I also turned on debugging but nothing showed when I tried to connect.
I'm sorry i can't help you. I'm using only the Oauth2 plugin ...
I just documented the process for setting up login with LinkedIn using the OAuth support added in Moodle 3.3.
See this page: https://docs.moodle.org/dev/OAuth2_Services_Setup_Project_In_LinkedIn#Setup_App_in_LinkedIn
If you switch to use this new service the first time your users login they will need to click a confirmation link in their email in order to connect to their existing Moodle accounts.
I followed your instructions to the letter, but I get this error message when I try to connect my account:
The system account was not connected for offline access
You don't need to connect a system account just for logins. The system account is only for things like repository plugins and file conversions (e.g. for google/office stuff).
Damyon, many thanks for your message.
I currently have moodle 3.2.3+ with previous "auth2" plugin.
Could you please help me with these tips?
1) I have many users created using "linkedin", "facebook" and Google+". If I update my moodle to 3.3+, will they be migrated to this new "Auth2 core"?
2) If thre's no automatic migration, is there a script "how-to" migrate?
Thanks in advance.
Best regards
Filipe
They will still be able to login - but the first time they do so with the new plugin they will be required to "confirm" that they own the email address by clicking a link in an email.
Damyon, good evening!
I tried to upgrade my moodle 3.2+ to 3.3+ and non of my services are working (Linkedin, Google and Facebook). Really really bad... could you please help me out? All I get is this message: "The system account was not connected for offline access". Urgent!
I'm right in the middle of upgrading from 3.1.7 to 3.3.1 and am also switching from using the googleoauth2 plugin to the new core OAuth 2 module. I can confirm that the process works as you've described above. I configured the new OAuth 2 Services with the Google client ID and secret, enabled it and disabled the old plugin. I immediately got a session expired message and had to log in again, and as you described, I received an email asking me to confirm that I wanted to link the account.
However, a new and major problem has emerged - sessions expire extremely quickly and I keep having to log in again. Basically, load a page, take a minute to read something, and you're kicked out. I couldn't even complete editing a forum post before I was getting error messages that drafts couldn't be saved.
Is there a setting somewhere that also needs to be updated? Because as it stands, this is unusable.
--- Les
Update: I've now done the following:
Updated Site Administration -> Server -> Session handling -> Timeout to 3 hours. (After being timed out once while doing so).
Cleared all site-related cookies.
Exited the Firefox browser and restarted
Completely rebooted the server.
Logged in from a different machine, using Chrome rather than Firefox.
Nothing worked. Within about a minute, I start to get error messages ("web service not available", "draft could not be saved", etc.) and the session times out.
I've gone back to the old googleoauth2 plugin for the time being, as most of my users prefer to authenticate via OAuth2. At least the system is usable once more, and I can research before, hopefully, switching back to the core module later.
Any suggestions gratefully welcomed.
--- Les
More test changes that didn't solve the problem:
The Google API client ID was set up with two authorised redirect URI's:
I created (actually, edited and reused an old) client ID that had only the latter redirect URI. Rebooted the server, cleared the cookies in the browser.
No joy.
--- Les
Hi Damyon,
I've migrated from v3.1 to v3.3.
In the old version I had installed and working with Google authentication the module Auth2.
After the migration I've configured the new core auth module and it's working fine.
The only problem (as said in this thread) before) is that I had to change in the database the field auth in the table mdl_user.
It was googleoauth2 in the previous version and I changed it by auth2.
As you say they receive an email, but system says that it's because the email address in not linked yet.
Is there any way to avoid this? It doesn't have any sense to these users because they've been working on the system with the google accounts for years.
I've checked the moodle database and I assume there's some relationship with a table called: mdl_auth_oauth2_linked_login.
Is there some way that me as Moodle administrator can link all the existing account in a massive way?
Thanks
PS. I assume this is happening to everybody who is upgrading the version and was using the old module, so it would be great some kind of solution for this.
Am just adding 2 cent opinion here .... I would hope that an Admin could NOT massively accomplish this for it would mean that individual (possibly all admins in a Moodle) could change controls/access to etc. for those individuals. Their accounts ... thus they need to be in control.
As an Admin of a Moodle I would NOT want to be taking on the security of their accounts ... unless all individuals involved were part of a Google Edu domain - and I was one of the Google Admins of that domain and authorized to control the Edu domain.
IF one looks at such things overall (not just Moodle) ... don't individual owners of an Android, as example, have control over granting permissions? It's becoming more common place with smartphones and tablets now so would think users may have already had a similar experience.
Making a change such as this in a Moodle site should be planned (understatement) and all users informed for weeks of a cutover/change date. Tutorials for such outside of Moodle with example screen shots, etc.
Yes, some will make a mistake ... prepare for it with another set of Tutorials/page info etc. on how to correct the problem.
Like I said ... 2 cent opinion.
Unless one can figure out how to provide Moodle with the following ... (seen in confirmation link):
site/auth/oauth2/confirm-linkedlogin.php?
a token
user id
username (which is the Email address)
issuer = 1 (which on the site being used has only Google auth configured).
And Moodle communicate to Google that it's OK.
'spirit of sharing', Ken
I'm just going through this at the moment - see my posts earlier in the thread, and since I was logged into MySQL anyway . . .
During my migration, I tested creating an account and logging in using a Google ID associated with my university email account. It created a row in mdl_user with the 'confirmed' column set to 0. While I was waiting for the confirmation email to turn up - it went into the spam folder - the account appeared in Site Administration -> Users -> Accounts -> Browse list of users with a "Confirm" link at the right end of the line, to allow manual confirmation.
When I got the email and went to the confirmation link, the 'confirmed' value went to 1.
I also looked at mdl_auth_oauth2_linked_login during this process - it has two columns called 'confirmtoken' and 'confirmtokenexpires', but they stayed blank right through the confirmation process, so I'm not sure how they're involved. Note, though, that this was for the creation of a new oauth2 user account, not migration of an existing googleoauth2 account.
So it might be as simple as
UPDATE mdl_user SET confirmed = 1 WHERE auth = 'oauth2';
Or it might not. ;)
I'm anxiously watching live logs to see if any of my users experience problems during the migration from googleoauth2 to oauth2, and if worse comes to worst, I might try the above statement as a way of "pre-authenticating" them to avoid any problems.
--- Les
Hi Les,
Please can I ask what the outcome of your test was and whether you got around existing users receiving an email asking them to authenticate their account on the switch from the OAuth Plugin to the Core OAuth?
Kind regards,
Sarah
