Moodle in English: RESOLVED: Hold off upgrading to Moodle 3.9.5 and 3.10.2

Hi All,

Apologies for cross-posting, as this is also posted in the general developer forum, but it is worthwhile repeating this message.

As reported in MDL-71156, there is an issue with an upgrade step, introduced in MDL-67494. Please also see the note in the last integration exposed post.

Please avoid upgrading to 3.9.5 or 3.10.2 until this issue is resolved.

During the upgrade some calendar events are corrupted.

I apologise for the inconvenience this may have caused for those sites who have already upgraded.

We are currently working on a fix for this issue and will update this discussion when that is available.

Sander

平均分：Useful (3)

Re: Hold off upgrading to Moodle 3.9.5 and 3.10.2

Diane Soini - 2021年03月23日星期二 04:20

What's the likelihood this issue might be fixed in the next 72 hours?

平均分： -

Re: Hold off upgrading to Moodle 3.9.5 and 3.10.2

Ken Task - 2021年03月23日星期二 07:07

@Sander ... Question ... for those who are using git, is there a 'dry-run' option to see what files/changes would be done in advance of pulling the trigger on updating?

Curious mind ... wanna know!

Thanks, in advance, for any response/advice ...

'SoS', Ken

平均分： -

Re: Hold off upgrading to Moodle 3.9.5 and 3.10.2

Matt T - 2021年03月23日星期二 18:41

That got added to git 2.27 last year.

git pull --dry-run

平均分： -

Re: Hold off upgrading to Moodle 3.9.5 and 3.10.2

Ken Task - 2021年03月23日星期二 21:52

Thanks, Matt! Hmmmm ...

not quite what I expected:

git version 1.8.3.1

[root@server moodle39]# git pull --dry-run
remote: Counting objects: 2179, done.
remote: Compressing objects: 100% (383/383), done.
remote: Total 2179 (delta 1762), reused 2176 (delta 1759)
Receiving objects: 100% (2179/2179), 810.10 KiB | 0 bytes/s, done.
Resolving deltas: 100% (1762/1762), completed with 571 local objects.
From git://git.moodle.org/moodle
   8ee6f49..2ad3d1a MOODLE_39_STABLE -> origin/MOODLE_39_STABLE
   6b804f9..2b26052 MOODLE_310_STABLE -> origin/MOODLE_310_STABLE
   295901e..c1db56b MOODLE_311_STABLE -> origin/MOODLE_311_STABLE
   fc335f5..4c26696 master     -> origin/master

Is above what your's looks like?

'SoS', Ken

平均分： -

Re: Hold off upgrading to Moodle 3.9.5 and 3.10.2

Matt T - 2021年03月23日星期二 22:07

Hm, sorry, while in your code directory try:

git diff origin/MOODLE_39_STABLE

(assuming you're on 3.9)

This should you the changed files. You can also run it on an individual file to see the changes in that file:

git diff origin/MOODLE_39_STABLE upgrade.php

平均分：Useful (1)

Re: Hold off upgrading to Moodle 3.9.5 and 3.10.2

Matt T - 2021年03月23日星期二 18:47

I could be naive, but at least in respect of core stuff, I would expect the calendar entries could just be recreated by code? Is there any real scope of data loss? It's not clear reading through the issue tracker.

We have already upgraded production, but fortunately we aren't affected, no rows were updated. I didn't bother rolling back to our 2.9.4+ backup.

I have been running weekly upgrades so far, but now might reassess if that's a good idea. Is it expected that people will upgrade reasonably soon after release to the production repository, or should I be waiting for real world testing to be completed?

平均分：Useful (2)

Re: Hold off upgrading to Moodle 3.9.5 and 3.10.2

Rick Jerz - 2021年03月23日星期二 21:47

Matt, might you share the SQL query that you ran to determine that none of your rows were updated? (I made a similar request in tracker.) When I run Eric's code that he posted in Tracker, I do get around 500 rows showing. However, even if I run this SQL query on an older Moodle, Moodle 3.10.1 (Build: 20210118), which is experimental and not upgraded to 3.10.2, I get about the same results.

Yes, I understand the concern about updating. As I recall, the notification about 3.10.2 was a "security update."

平均分：Useful (1)

Re: Hold off upgrading to Moodle 3.9.5 and 3.10.2

Matt T - 2021年03月23日星期二 22:11

I assume I ran the same one you did:

SELECT * FROM mdl_event WHERE (eventtype <> 'user' OR priority <> 0) AND courseid = 0 AND groupid = 0 AND categoryid = 0;

Alternatively

SELECT COUNT(*) FROM mdl_event WHERE courseid = 0 AND userid = 0 AND groupid = 0 AND categoryid = 0;

If you want to compare your pre and post backup, then SELECT id,modulename,userid (...) will probably get you the important data - perhaps put both in text files and import them into MS Excel or equivalent for comparison.

I imagine the fact you've already got rows for the query would be good information for the developers - I see you've mentioned it in the issue tracker.

平均分：Useful (1)

Re: Hold off upgrading to Moodle 3.9.5 and 3.10.2

Rick Jerz - 2021年03月23日星期二 22:58

Thanks Matt. Yep, Eric replied over in Tracker with similar thoughts and advice.

平均分： -

Re: Hold off upgrading to Moodle 3.9.5 and 3.10.2

Sander Bangma - 2021年03月23日星期二 20:44

Hi all,

I want to update you on the current status and action plan.

We are working hard to investigate the issue and recover any potentially lost calendar events. We will take a 2-step approach:

We will prepare an additional minor release (Moodle 3.9.6 and 3.10.3) which resolves the regression by reverting MDL-67494. The aim here is to provide a release that is safe to enable sites to upgrade to the latest minors. We are currently confirming there are no unexpected side effects from reverting and anticipate to be able to release these minors by Thursday or Friday this week.
We are reviewing all possible scenarios where events might have been corrupted, with the aim to provide and release a solution that restores all affected calendar events. This will take a bit longer and we will provide further updates on this as we learn more.

If you have not upgraded your Moodle site to 3.9.5 or 3.10.2 yet, please avoid upgrading until we have released the fixed versions (Moodle 3.9.6 and 3.10.3), so that you remain unaffected by this bug.

If you have already upgraded to one of the affected versions, we apologise for the inconvenience this may have caused. We will provide further updates as we progress with the solution to recover any potentially affected events.

平均分：Useful (7)

Re: Hold off upgrading to Moodle 3.9.5 and 3.10.2

Sander Bangma - 2021年03月25日星期四 10:45

Further to this;

We have now released Moodle 3.9.6 and 3.10.3 which revert MDL-67494. These minors provide a safe upgrade pathway.

Work on a solution to recover any potentially affected calendar events is under way, please also follow the discussion in MDL-71156.

You can read more details in the news post on moodle.org.

平均分：Useful (1)

Re: Hold off upgrading to Moodle 3.9.5 and 3.10.2

Natassia Stelmaszek - 2021年03月26日星期五 00:49

I just tried updating to 3.10.3 and got:

sudo php admin/cli/upgrade.php --non-interactive
No upgrade needed for the installed version 3.10.2+ (Build: 20210312) (2020110902.01). Thanks for coming anyway!

I thought that this needed to be patched???

Natassia

平均分： -

Re: Hold off upgrading to Moodle 3.9.5 and 3.10.2

Ken Task - 2021年03月26日星期五 02:00

Site installed using git? If so, one would have to run git pull to acquire the 3.10.3 code and then the upgrade.php script.

'SoS', Ken

平均分： -

Re: Hold off upgrading to Moodle 3.9.5 and 3.10.2

Rick Jerz - 2021年03月26日星期五 20:04

In the "for what it is worth" category, this morning I finished fixing my existing problem. (Many thanks to Eric and others.)

I am going to attach my notes to what I did, which was basically doing an UPDATE query of my production Moodle from a backup Moodle. (The file is actually a text file with comments and SQL).

I also made this video showing what I did, which might help some.

Fix_Regression.sql

平均分：Useful (4)

Re: Hold off upgrading to Moodle 3.9.5 and 3.10.2

Matt T - 2021年03月26日星期五 20:50

Well done! I'm glad you were able to fix it and no doubt your instructions will be valued.

It might be worth nothing that for users who...

A) are only using core activities (no third party activity plugins); and

B) are not comfortable with SQL

...it /may/ be a better idea to wait for the Moodle stage 2 patch, which will (in theory) gracefully recreate the calendar entries. Of course, this will also require some patience on the part of those users.

That said, a lot of people will surely find this helpful. Good work.

平均分： -

Re: Hold off upgrading to Moodle 3.9.5 and 3.10.2

Rick Jerz - 2021年03月27日星期六 09:52

Yes, stage 2 will be interesting. I will be curious to see this fix. I am confident that it can be done, just as I was able to see that I did something to Ray.

The query could be something like: Find all students who have a user override but are not found in the events table.

平均分： -

Re: Hold off upgrading to Moodle 3.9.5 and 3.10.2

Halim Asa - 2021年03月28日星期日 18:46

I have not encountered this problem, I have not yet developed my site enough to use the activities, I continue to prepare the contents. For a course that will last about 40 hours, I do both text research and create the learning method. Lessons, section, cluster and questions ...

And I watched the video you created for the solution. I've marked it as useful. Thank you very much for the work

However, when I saw @Sander's message, I took my backups and updated my moodle version.
I noticed something like this here, if you know, can you answer?

I use 4 plugins in my moodle system, course stats, attandence, customcertificate etc.
When I try to update my Moodle versions, I delete all the moodle installation files. I'm backing up the moddledata folder. When I load the new moodle files and put the old config file into it, it goes into the installation and says that the files of these 4 plugins are not found. Next to it is the update button to install this add-on. But am I doing it wrong? Do I need to back up the plugins in the moodle upload file besides the moodledata folder?

平均分： -

Re: Hold off upgrading to Moodle 3.9.5 and 3.10.2

Rick Jerz - 2021年03月29日星期一 09:08

You say that you "delete all the moodle installation files." In this step, my guess is that you are also deleting plugins, too.
Then, you load the new moodle and retain your moodledata files. If this is true, your Moodle database thinks that these plugins should be there, but they are not, and this gives you error messages.

My upgrade method is similar. But after I install the new Moodle, and move the config file, I manually reinstall the plugins. What I mean by "manually install" is that I download the plug-in, unzip it (in my downloads folder), then move this into the correct Moodle folder. For example, I know that Collapsed topics goes into admin | course | formats.

I don't reinstall the plugins from a backup, I always download the latest version to match my Moodle version, since sometimes plugins can be updated.

Hope this makes some sense to you. I think I show this in one of my videos, located in another topic. What I show in the update video is using cPanel. I made this video as a response to Ken. My actual upgrade is a little different, yet similar in principal.

平均分： -

Re: Hold off upgrading to Moodle 3.9.5 and 3.10.2

Halim Asa - 2021年03月29日星期一 22:03

Two nice responses, ☺️ ☺️ Thank you @ Rick and @Nurbek I realized that the upgrade I did was correct.
Thank you so much.
@Nurbek I will have a question for you when I make a backup of the mod and block folders, I will have thrown the outdated add tielri. Is it necessary to keep the add-on folders by saying whether it is necessary or not.

I thought there was no need to backup the ekelnti folder. In Rick's reply. Since the settings are already in the database when the latest version of the addon is installed, it will continue to work with the same settings when updated plugin files are available.

平均分： -

Re: Hold off upgrading to Moodle 3.9.5 and 3.10.2

Rick Jerz - 2021年03月29日星期一 23:59

I don't know about ekelnti. If it is a plugin, treat it like your other plugins.

Another suggestion is not to delete your original Moodle. Instead, rename it to something like moodlex. Then, you will have the ability to "copy" (not move) old items to your new moodle. When your new Moodle is running and proved to be good, then you can delete moodlex.

平均分： -

Re: Hold off upgrading to Moodle 3.9.5 and 3.10.2

Nurbek Tentiev - 2021年03月30日星期二 00:07

You don't need to have old, i.e. outdated plugins in the new system's folder. Use latest plugins available for the Moodle version you have chosen for the upgrade. If the plugins are standard ones (downloaded from the Moodle official web site), I would suggest checking https://moodle.org/plugins to see whether new versions are available. Download and put them instead of previous/old ones.

I have no idea on the plugins you are referring to - tielri and eklenti. There are no such ones in the standard directory. Perhaps they are your own developed modifications/add-ons. If so, you should know if they will be compatible.

I would suggest making backups of everything. This is kind of golden rule for any upgrade. Just in case...

平均分： -

Re: Hold off upgrading to Moodle 3.9.5 and 3.10.2

Nurbek Tentiev - 2021年03月29日星期一 19:27

Before upgrading the system you need to place plugins' folders to correct folders of Moodle. For example, they might be in 'mod' or 'block' folders. Also make sure that you've put the plugins that are compatible with the moodle version.

平均分： -

Re: Hold off upgrading to Moodle 3.9.5 and 3.10.2

Jiří Procházka - 2021年03月24日星期三 18:25

I already updated production site to 3.9.5+ successfully. I can not roll back, it is too late. We are using calendar only for internal events, because imported Google calendars are displayed incorrectly at all if there are events longer than one day. There is a problem with interpretation of time 0.00.00 and day, so for example holidays from 1st to 5th of April are displayed as from 1st to 6th of April

Using external calendars in Moodle is more dangerous than this particular bug

平均分： -

Re: Hold off upgrading to Moodle 3.9.5 and 3.10.2

Ken Task - 2021年03月25日星期四 07:25

I see in a 3.10.x git pull an update to 3.10.3 .. ...

Some calendar + other ... clip:

calendar/classes/privacy/provider.php                                    | 167 ++++++++++++++++++++++------
calendar/lib.php                                                         | 131 +++++++---------------
calendar/tests/event_vault_test.php                                      | 56 +++++-----
calendar/tests/externallib_test.php                                      | 74 ++++++-------
calendar/tests/local_api_test.php                                        | 12 +-
calendar/tests/privacy_test.php

is this the fix?

'SoS', Ken

平均分： -

Re: Hold off upgrading to Moodle 3.9.5 and 3.10.2

Dan Bartlett - 2021年04月29日星期四 16:10

Hi,

Any further news on this? We currently have 3.9.5 on live production and would prefer not to upgrade if possible since we've implemented a change freeze as a lot of load testing for exams was done on this version. Is there is any risk of this bug causing Quizzes/Assignments becoming visible at the wrong time (or not showing at the right time) - really not something want to happen over exams. Also does it only affect Calendar events created before 3.9.5 or events created since then too?

I've tried running the 3.9.6 fix script on our demo site and it was only partially successful (basically couldn't do the last group of 'Other'). I wondered of the feasibilty of doing an update using the userid of when the event was created in logstore table (objecttable 'event', objectid event.id and action 'created'? May be problem with timeouts etc on updates using such a large table though.

Any news on progress of this would be really appreciated, as well as advice on whether to make changes just before sensitive/important exam time.

Dan

平均分： -

Re: Hold off upgrading to Moodle 3.9.5 and 3.10.2

Luis de Vasconcelos - 2021年04月30日星期五 16:52

It seems like the problem has been fixed. You can see the details on MDL-71156.

平均分： -

Re: Hold off upgrading to Moodle 3.9.5 and 3.10.2

Dan Bartlett - 2021年04月30日星期五 21:58

Thank you for helping. Just wondered if anyone knew if there's a risk to visibilty of modules breaking while still on 3.9.5. (showing or hiding when they shouldn't).

平均分： -

Re: Hold off upgrading to Moodle 3.9.5 and 3.10.2

Gerald Crumbley - 2021年05月7日星期五 22:49

If this issue has been resolved, Maybe it should be removed from "Pinned" at the top of the forums, or edited to say "Resolved"

平均分： -

Re: Hold off upgrading to Moodle 3.9.5 and 3.10.2

Sander Bangma - 2021年05月9日星期日 11:00

Thanks Gerald, done both of those now

平均分： -