Unable to "Upgrade Moodle Database Now"

Unable to "Upgrade Moodle Database Now"

by Hariharasudhan A S -
Number of replies: 8

Hi!

Ever since I tuned my database server to accept high concurrent users (using mysql tuner), I am unable to install plugins or upgrade moodle. I always get stuck on the Plugin Check page where you have to click on "Upgrade Moodle Database Now". It usually takes an unusually long time and then shows 502 Gateway error. Trying to purge all caches too takes a long time and then shows the 502 page. I am not sure if Database is the problem here.

MySQL error log:


2020-11-11  0:29:13 23881467 [Warning] Aborted connection 23881467 to db: 'moodle' user: 'moodle
user' host: '95.216.221.150' (Got an error reading communication packets)

PHP-fpm log:

[11-Nov-2020 00:29:13] WARNING: [pool www] child 903 exited on signal 9 (SIGKILL) after 4897.868
787 seconds from start

Nginx error log:

2020/11/11 00:29:13 [error] 857#857: *11843 recv() failed (104: Connection reset by peer) while
reading response header from upstream, client: 10.0.0.5, server: sathyabama.cognibot.in, request
: "GET /admin/index.php?cache=0&confirmrelease=1&confirmplugincheck=1 HTTP/2.0", upstream: "fast
cgi://127.0.0.1:9000", host: "sathyabama.cognibot.in", referrer: "https://sathyabama.cognibot.in
/admin/index.php?cache=0&confirmrelease=1&confirmplugincheck=0"

The webserver, database, redis and filestore are each in seperate servers. They all run Debian and Moodle 3.9.1.

Please let me know how to go about solving this and if you need any additional information. Thanks!

Average of ratings: -
In reply to Hariharasudhan A S

Re: Unable to "Upgrade Moodle Database Now"

by Ken Task -
Picture of Particularly helpful Moodlers

Have you tried command line upgrade from script in admin/cli/ of code?   That reads config.php values and really doesn't use web service interface so it's just php talking to DB server.

Will still show errors.

Of course, always a good idea to get a db dump prior to updating/upgrading anything ... by any method! smile

Also ... https://www.percona.com/blog/2016/05/16/mysql-got-an-error-reading-communication-packet-errors/

You might need to tweak some settings as updates and upgrades can involve lots of data!

'SoS', Ken


Average of ratings: Useful (1)
In reply to Ken Task

Re: Unable to "Upgrade Moodle Database Now"

by Hariharasudhan A S -
Thanks for the reply. I tried upgrading moodle by copying the new Moodle 3.10 code to my /var/www folder following the instructions here. The same problem occurs in that case too. The first time I logged in as admin to the updated server, it takes me to the "Upgrade Moodle Database Now" page and gets stuck as usual. 

> Also ... https://www.percona.com/blog/2016/05/16/mysql-got-an-error-reading-communication-packet-errors/

>Have you tried command line upgrade from script in admin/cli/ of code?

Will check these out and update here. Thank you.
Average of ratings: -
In reply to Hariharasudhan A S

Re: Unable to "Upgrade Moodle Database Now"

by Visvanath Ratnaweera -
Picture of Particularly helpful Moodlers Picture of Translators
I think the culprit is this:
> PHP-fpm log:
> [11-Nov-2020 00:29:13] WARNING: [pool www] child 903 exited on signal 9 (SIGKILL) after 4897.868
> 787 seconds from start

Where does php-fpm run? Check its logs. If not enough information, raise raise the debug level.

If crashing php-fpm is the cause, your problem is not in Moodle.
Average of ratings: Useful (1)
In reply to Visvanath Ratnaweera

Re: Unable to "Upgrade Moodle Database Now"

by Hariharasudhan A S -
Thanks for your reply.

>If not enough information, raise raise the debug level.

Tried this - set the debug level to developer. Found out that pm.max_children was reaching its limit. Increased the limit (doubled it) which led to around 30 mins of wait time after clicking on "Upgrade Moodle Database Now" button (Installing a simple plugin - a process which normally took around 30s a few months back).

This is the php error log now. (Didn't want to clutter the forum, so on pastebin) Got 504 Gateway error after around 30 mins.

Also, after increasing pm.max_children I am not getting any error in mysql server. 30 mins seem like an absurdly long time for a PHP process to run (especially one like installing a plugin), so I am sceptical as to if PHP is the issue here. I am stuck as to where to go from here. Any help is appreciated. Thank you.
Average of ratings: -
In reply to Hariharasudhan A S

Re: Unable to "Upgrade Moodle Database Now"

by Jerry Lau -
start a clean install of both the application and database you use...

do that first and test it out...
Average of ratings: -
In reply to Hariharasudhan A S

Re: Unable to "Upgrade Moodle Database Now"

by Jerry Lau -
You can ignore such messages...because they are NOTICES.
They may appear because someone have set the pm.max_request in pool configuration file
/etc/phpX/fpm/pool.d/www.conf

It's the number of requests each child process should execute before respawning. Default value is "0".
Average of ratings: Useful (1)