Upgrade 2.9+ to 3.0.10

Upgrade 2.9+ to 3.0.10

by Murilo Saldanha -
Number of replies: 8

Hi to all!

I'm trying to update Moodle but after clicking the button to update the Moodle database, an error occurs as shown in the attached image.



my system:

Ubuntu 16.04
PHP 7.0
Nginx
postgres (9.3.8)

Moodle
$version  = 2015051100.07;
$release  = '2.9+ (Build: 20150618)'


Average of ratings: -
In reply to Murilo Saldanha

Re: Upgrade 2.9+ to 3.0.10

by Visvanath Ratnaweera -
Picture of Particularly helpful Moodlers Picture of Translators
Where is this Update button? If it is belongs to an administration interfce offered by your hosting provider, you have to file a complaint with them.

There are few suspicious things in your setup:
- How come you are still at https://docs.moodle.org/dev/Releases#Moodle_2.9 ? The last security update is almost four years ago!


- And why is the target release 3.0? If your aim is to march your site to a current version of Moodle, the next milestone should be 3.1 or 3.2.
In reply to Visvanath Ratnaweera

Re: Upgrade 2.9+ to 3.0.10

by Murilo Saldanha -
hi Visvanath, grateful for your answer

Where is this Update button? If it is belongs to an administration interfce offered by your hosting provider, you have to file a complaint with them.
the button I mentioned is part of the moodle update screens, plugins verification screen

How come you are still at https://docs.moodle.org/dev/Releases#Moodle_2.9 ? The last security update is almost four years ago!
unfortunately this service was not under my administration, however I am taking over now and I intend to organize things.

And it was not PHP 7.0 compatible either https://docs.moodle.org/dev/Moodle_2.9_release_notes#Server_requirements.
I followed your suggestion and installed PHP 5.6 and the problem persisted.

And why is the target release 3.0? If your aim is to march your site to a current version of Moodle, the next milestone should be 3.1 or 3.2.
I upgraded to the latest version of 2.9, which is 2.9.9. The next step was to upgrade to version 3.1, it didn't work. I did a restore to version 2.9.9 and tried to update to 3.2, it also didn't work. The error output is this:

Attachment error_update_database_moodle_2.9-3.1.png
In reply to Murilo Saldanha

Re: Upgrade 2.9+ to 3.0.10

by Visvanath Ratnaweera -
Picture of Particularly helpful Moodlers Picture of Translators
Hi

To avoid a misunderstanding:
> the button I mentioned is part of the moodle update screens, plugins verification screen

So it is the second part of an upgrade, Moodle updating the database, etc. The first part replacing the old software with the newer version, you have done manually. Right?

> I followed your suggestion and installed PHP 5.6 and the problem persisted.

> I upgraded to the latest version of 2.9, which is 2.9.9. The next step was to upgrade to version 3.1, it didn't work. I did a restore to version 2.9.9 and tried to update to 3.2, it also didn't work.

So you have clean and running 2.9.9. Can't move forward, neither to 3.1 nor 3.2, the error you attached appears?

I can't remember this alteration to the database, but it is exactly the kind of changes Moodle does during upgrades. On the face of it, the ALTER TABLE hits a time out. How big, how many records, is the table mdl_message? What are your hardware resources? Are you aware of https://docs.moodle.org/en/Administration_via_command_line#Upgrading ?
In reply to Visvanath Ratnaweera

Re: Upgrade 2.9+ to 3.0.10

by Murilo Saldanha -
>So it is the second part of an upgrade, Moodle updating the database, etc. The first part replacing the old software with the newer version, you have done manually. Right?

exactly!


>So you have clean and running 2.9.9.

I am considering that the expression "clean" means running without errors and not clean data, ok? Yes, version 2.9.9 run clean and without errors.

>Can't move forward, neither to 3.1 nor 3.2, the error you attached appears?

Yes

> On the face of it, the ALTER TABLE hits a time out.
PHP script execution time has been increased to 300. Is there some similar configuration to increase the timeout of queries for postgres?


> How big, how many records, is the table mdl_message?

relation | total_size
---------------------------+------------
mdl_logstore_standard_log | 38 GB
mdl_grade_items_history | 4952 MB
mdl_stats_daily | 2250 MB
mdl_message | 1439 MB
mdl_grade_grades_history | 1326 MB
(5 rows)

count
--------
457181
(1 row)

is the fourth largest table with 1439 Mb and 457,181 lines

a curiosity...
In the first attempt to update, moodle 2.9 to 3.0.10, the mdl_logstore_standard_log table was the problem. It is the largest table with 38 GB.

>What are your hardware resources?
moodle, moodle chat and postgree are on different VMs and I can increase capacity as needed.

VM Postgres Database
-2 CPUs
-8 Gb Ram
-200 GB Storage

VM Moodle
-4 CPUs
-16 Gb Ram
-100 GB Storage

VM Moodle Chat
-1 CPUs
-4 Gb Ram
-20 GB Storage



>Are you aware of https://docs.moodle.org/en/Administration_via_command_line#Upgrading ?

I know about the possibility of installing via CLI but would it be useful in that case?
In reply to Murilo Saldanha

Re: Upgrade 2.9+ to 3.0.10

by Visvanath Ratnaweera -
Picture of Particularly helpful Moodlers Picture of Translators
Aha! Now we arrived at the cause of the problem. Before moving any further, do you really want to migrate 38 GB of log data, which sit in the database, BTW? I think it is prohibitive. On top of it, over the years the mechanism changed, a new one was introduced. So I question the use of the old data.

I hope, somebody, who knows the Moodle's logging mechanism better, will come to the rescue. I remember recommendations to just delete them, or delete the records older than a certain age.
In reply to Visvanath Ratnaweera

Re: Upgrade 2.9+ to 3.0.10

by Leon Stringer -
Picture of Core developers Picture of Particularly helpful Moodlers
Logs

By default Moodle keeps logs forever which causes the database table to grow. In my experience you don't need to keep logs for more than one year but you need to decide this (your organisation may have a records retention policy). You can change this in Keep logs for under Site administrationPluginsLoggingStandard log.

But if there are lots of old entries removing these can take (many) days. They're removed by the cron, it deletes as many old rows as it can for five minutes every day.

So it might be easier to remove these from the database table directly. E.g. to delete log entries older than a year:

=# DELETE FROM mdl_logstore_standard_log WHERE timecreated < EXTRACT(EPOCH FROM TIMESTAMP '2019-08-21');

Ensure you have a backup of the database before making changes like this.

Error: Canceling statement due to statement timeout

This is a PostgreSQL error so increasing max_execution_time won't fix this. You can check what this is set to in the PostgreSQL shell with SHOW statement_timeout.

=# SHOW statement_timeout;
 statement_timeout
-------------------
 0

I think this is 0 (i.e. no limit) by default so maybe it's been enabled on your system? You can change this in postgresql.conf but it will affect other users and databases if Moodle isn't the only database on the database server.



Average of ratings: Useful (1)
In reply to Murilo Saldanha

Re: Upgrade 2.9+ to 3.0.10

by Murilo Saldanha -
hello Visvanath and Leon! next few days I will be working on database questions and as soon as I have answers, negative or positive, I will post again.

thank you for help
In reply to Murilo Saldanha

Re: Upgrade 2.9+ to 3.0.10

by Murilo Saldanha -
hello Leon.

The commands helped to solve the problem. I used too VACUUM was used command as part of the solution, which is responsible for performing maintenance on the tables and permanently delete the rows that were marked by DELETE, effectively reducing the table size.

Once again I thank you and Visvanath for help.
Average of ratings: Useful (1)