Baffling upgrade issue...

Baffling upgrade issue...

by Emma Richardson -
Number of replies: 24
Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Plugin developers
Ok, I will put the full story here because maybe, if nothing else, it will help me figure it...Ken has been troubleshooting with me and we are still at a loss so thought I would try and get some eyes on this.

Several months ago, I tried to upgrade from 4.1 to 4.3 - I got a horrible white screen issue - I decided it must be php because that is what white screen normally means (lol, that is what I always tell everyone else)...after fighting it a while and not getting anywhere, I decided it would be easier to build a new server with php 8.1 and migrate the site across.

This worked great for my 4.1 site which has been running quite happily on the new server for a few months.

Today, I thought, well I should go ahead and upgrade now....and to my despair, I have the same white screen - now it doesn't even try to let me upgrade- as soon as I pull down the 4.3 code via git, the white screen of death appears and if I try and run upgrade.php through cli, it does nothing, just jumps back to a command prompt like it has completed but no output and nothing happens...I go back to 4.1 and all is good again.  I try 4.2 and the same thing happens as with 4.3.

I have poured through logs and fixed every error that I could find and my logs now look cleaner that they have ever done!  I have deleted all of my moodledata (except for filedir) but had to restore the lang directory as apparently that held all my language customizations...I have run checks.php and nothing there - environment also shows my server to be fully compatible.

I turned on full developer debugging and run the upgrade.php to see if that pulled any errors but it still does absolutely nothing.  This is on a debian server 5.10.97.  DB is on another server running on Mariadb.  No errors on that server.

I have uninstalled and deleted the assignment folder.  Which promptly reinstalls everytime i switch back to 4.1 branch!  What baffles me is that it won't even try to install from the command line - which tends to rule out apache issues...

Would appreciate any ideas of where to look next...could a plugin cause this behavior - I seem to think if it was a plugin it would cause issues before the install, not stop it from even starting...but maybe I am wrong... I tell everyone else that this is a php issue but I am not finding anything wrong with my php install at this point...

I just decided to try with a new config file - that allowed the site installer to load (progress!!) - I got to the lang page (the page after entering db details) and it whitescreened there...but just to reiterate that the site connects to the db just fine with 4.1...
Average of ratings: -
In reply to Emma Richardson

Re: Baffling upgrade issue...

by Visvanath Ratnaweera -
Picture of Particularly helpful Moodlers Picture of Translators
The log trace from Debugging?

> I have uninstalled and deleted the assignment folder. Which promptly reinstalls everytime i switch back to 4.1 branch!

mod/assign or mod/assignment? Anything to do with the old Assignment code gone?
In reply to Visvanath Ratnaweera

Re: Baffling upgrade issue...

by Ken Task -
Picture of Particularly helpful Moodlers
Think we discovered that the uninstall_plugins.php script in /code/admin/cli/ when used on 'assignment' does work and removes the tables in the DB related + mdl_config_plugins table, but doesn't remove the assignment directory in mod.   Manually removing mod/assignment will prevent re-install of the plugin.   That core plugin presence will halt a 4.3 upgrade.

'SoS', Ken

Average of ratings: Useful (1)
In reply to Visvanath Ratnaweera

Re: Baffling upgrade issue...

by Emma Richardson -
Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Plugin developers
No error messages with debugging on.
Assignment. Which has to be removed per instructions.
In reply to Emma Richardson

Ri: Baffling upgrade issue...

by Sergio Rabellino -
Picture of Particularly helpful Moodlers Picture of Plugin developers
Dear Emma, yes, some types of third party plugins can break moodle. It depends on what they are supposed to do and when in the logic are “executed”.
I would take a look at blocks, local plugins and into themes…
Even a single removed function from the core, but used in a third party (not  coherently updated) plugin could be the main cause.
hth
In reply to Sergio Rabellino

Re: Ri: Baffling upgrade issue...

by Emma Richardson -
Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Plugin developers
That is what I am scared of - with 56 additional plugins, it is going to take me a while to figure this one out...
In reply to Emma Richardson

Ri: Re: Ri: Baffling upgrade issue...

by Sergio Rabellino -
Picture of Particularly helpful Moodlers Picture of Plugin developers
Please verify that in the php.ini of you php cli (use php -i to view what are the files used)  the display_errors is set to "on", or php does not show the syntax error at run time that occurs.
These are different errors and are managed in a different way from the usual errors which are under the log_errors directive.
HTH
In reply to Sergio Rabellino

Re: Ri: Re: Ri: Baffling upgrade issue...

by Emma Richardson -
Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Plugin developers
Yes, I did that have set on and it didn't bring anything up.
In reply to Emma Richardson

Ri: Re: Ri: Re: Ri: Baffling upgrade issue...

by Sergio Rabellino -
Picture of Particularly helpful Moodlers Picture of Plugin developers
No, somehere (maybe over the rainbow …) there should be some syntax erroor logged.
In reply to Sergio Rabellino

Re: Ri: Re: Ri: Re: Ri: Baffling upgrade issue...

by Emma Richardson -
Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Plugin developers
If only I could find that! Guess I will keep looking - would be a lot easier if I could figure out the error instead of rebuilding a duplicate server...
In reply to Emma Richardson

Re: Ri: Baffling upgrade issue...

by Ken Task -
Picture of Particularly helpful Moodlers
Ok, we're down to plugins.

Moodle core doesn't have a 'check plugins' tool/script.

So ..... 'roll your own' ...  live server example:

Use uninstall_plugins.php script in admin/cli/ to get a list of plugins:

php uninstall_plugins.php --show-contrib will list them
Use same to create a addons.txt file like:

php uninstall_plugins.php --show-contrib > addons.txt

Edit addons.txt so that you have only the short names of the addons:

mod_googlemeet  Google Meet™ for Moodle
mod_hvp H5P
mod_offlinequiz Offline Quiz
offlinequiz_correct     Offlinequiz Results Correct
offlinequiz_overview    Offlinequiz Results Overview
offlinequiz_regrade     Offlinequiz Regrading
offlinequiz_rimport     Offlinequiz Results Import
offlinequiz_statistics  Offline Quiz Statistics
block_quickmail Quickmail
qtype_wq        Wiris Quizzes
filter_wiris    MathType by WIRIS
atto_morefontcolors     More font colors
atto_wiris      MathType by WIRIS
tiny_wiris      MathType by WIRIS
tool_userdebug  User related debugmodus
local_deleteoldquizattempts     Old quiz and question attempts deletion
local_mailtest  eMail Test

Becomes

mod_googlemeet
mod_hvp
mod_offlinequiz
offlinequiz_correct
offlinequiz_overview
offlinequiz_regrade
offlinequiz_rimport
offlinequiz_statistics
block_quickmail
qtype_wq
filter_wiris
atto_morefontcolors
atto_wiris
tiny_wiris
tool_userdebug
local_deleteoldquizattempts
local_mailtest

Install moosh

Then create a bash shell script that will loop through the addons.txt file
and use moosh to check moodle.org plugins.

checkaddons script

while read i
do
    echo 'Shortname of plugin:'$i;
    moosh -n plugin-list |grep $i
done < ./addons.txt

run the script: source ./checkaddons

Output:

Shortname of plugin:mod_googlemeet
mod_googlemeet,3.10,3.11,3.7,3.8,3.9,4.0,4.1,4.2,https://moodle.org/plugins/download.php/29096/mod_googlemeet_moodle42_2023050101.zip
Shortname of plugin:mod_hvp
mod_hvp,2.5,2.6,2.7,2.8,2.9,3.0,3.1,3.10,3.11,3.2,3.3,3.4,3.5,3.6,3.7,3.8,3.9,4.0,4.1,4.2,4.3,https://moodle.org/plugins/download.php/30739/mod_hvp_moodle43_2023122500.zip
Shortname of plugin:mod_offlinequiz
mod_offlinequiz,2.6,2.7,2.8,3.1,3.10,3.11,3.2,3.3,3.4,3.5,3.6,3.7,3.8,3.9,4.0,4.2,4.3,https://moodle.org/plugins/download.php/31235/mod_offlinequiz_moodle43_2024012200.zip
Shortname of plugin:offlinequiz_correct
Shortname of plugin:offlinequiz_overview
Shortname of plugin:offlinequiz_regrade
Shortname of plugin:offlinequiz_rimport
Shortname of plugin:offlinequiz_statistics
Shortname of plugin:block_quickmail
block_quickmail,2.0,2.1,2.2,2.3,2.4,2.5,2.6,2.7,2.8,2.9,3.0,3.1,3.2,3.3,3.4,3.5,3.7,3.8,3.9,4.1,4.2,https://moodle.org/plugins/download.php/29700/block_quickmail_moodle42_2023070700.zip
block_quickmailsms,2.3,https://moodle.org/plugins/download.php/1563/block_quickmailsms_moodle23_201210081202.zip
Shortname of plugin:qtype_wq
qtype_wq,3.0,3.1,3.10,3.11,3.2,3.3,3.4,3.5,3.6,3.7,3.8,3.9,4.0,4.1,4.2,4.3,https://moodle.org/plugins/download.php/30990/qtype_wq_moodle43_2024010801.zip
Shortname of plugin:filter_wiris
filter_wiris,2.2,2.3,2.4,2.5,2.6,2.7,2.8,2.9,3.0,3.10,3.1,3.11,3.2,3.3,3.4,3.5,3.6,3.7,3.8,3.9,4.0,4.1,4.2,4.3,https://moodle.org/plugins/download.php/30675/filter_wiris_moodle43_2023121300.zip
Shortname of plugin:atto_morefontcolors
atto_morefontcolors,2.7,2.8,2.9,3.0,3.1,3.10,3.11,3.2,3.3,3.4,3.5,3.6,3.7,3.8,3.9,4.0,4.1,4.2,https://moodle.org/plugins/download.php/24505/atto_morefontcolors_moodle42_2021062100.zip
Shortname of plugin:atto_wiris
atto_wiris,2.7,2.8,2.9,3.0,3.10,3.1,3.11,3.2,3.3,3.4,3.5,3.6,3.7,3.8,3.9,4.0,4.1,4.2,4.3,https://moodle.org/plugins/download.php/30676/atto_wiris_moodle43_2023121300.zip
Shortname of plugin:tiny_wiris
tiny_wiris,2.4,2.5,2.6,2.7,2.8,2.9,3.0,3.1,3.10,3.11,3.2,3.3,3.4,3.5,3.6,3.7,3.8,3.9,4.0,4.1,4.2,4.3,https://moodle.org/plugins/download.php/30678/tiny_wiris_moodle43_2023121300.zip
Shortname of plugin:tool_userdebug
tool_userdebug,3.10,3.11,3.7,3.8,3.9,4.0,4.1,4.2,4.3,https://moodle.org/plugins/download.php/30832/tool_userdebug_moodle43_2024010900.zip
Shortname of plugin:local_deleteoldquizattempts
local_deleteoldquizattempts,2.7,2.8,2.9,3.0,3.1,3.10,3.11,3.2,3.3,3.4,3.5,3.6,3.7,3.8,3.9,4.0,4.1,4.2,4.3,https://moodle.org/plugins/download.php/30851/local_deleteoldquizattempts_moodle43_2024011400.zip
Shortname of plugin:local_mailtest
local_mailtest,2.4,2.5,2.6,2.7,2.8,2.9,3.0,3.1,3.10,3.11,3.2,3.3,3.4,3.5,3.6,3.7,3.8,3.9,4.0,4.1,4.2,4.3,https://moodle.org/plugins/download.php/31219/local_mailtest_moodle43_2024022200.zip

The urls to the listing shows highest version and the link to the plugin .zip file
for that highest version of the plugin.   If we are trying to upgrade from 4.1 to anything higher like 4.2 or 4.3 any plugin you have that isn't compat with those higher versions could be a problem.  Example: the first mod for google meet shows up to 4.2.   If I take this site to 4.3 that mod may not function and/or could be a cause for upgrade failure.

Note: a lot of times (been my experience) addons can work when core is one higher version upwards without issues - if you are lucky! smile

Comment: the good thing about moodle is that you can install addons!  the bad thing about moodle is that you can install addons!

'SoS', Ken

In reply to Emma Richardson

Re: Baffling upgrade issue...

by Rick Jerz -
Picture of Particularly helpful Moodlers Picture of Testers
What happens if you try to install a brand new Moodle, manually, meaning not using GIT?
 
For what it is worth, I am running Moodle 4.3.3+ (Build: 20240223) using PHP 8.0.30 on my server and PHP 8.0.8 in MAMP.  My server is using MySQL 8.0.35 , whereas MAMP is using MySQL 5.7.34.
In reply to Rick Jerz

Re: Baffling upgrade issue...

by Emma Richardson -
Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Plugin developers
Pretty sure I have determined that this is not a php issue as the installer will at least start when I remove the config file.
In reply to Emma Richardson

Re: Baffling upgrade issue...

by Ken Task -
Picture of Particularly helpful Moodlers
When Emma and I communicated earlier, she forgot to mention the DB server was remote.
Works in 4.1 but can't upgrade because upgrade.php cli script goes to white screen immediately has nothing to do with how code was acquired ... the DB user privlieges do, however.

So, Emma .. what are the grants on the DB user in the config.php file of the site?
If root user is allowed remote access, could add 2 lines in config that change the user to root with root password and then run the upgrade.php script.

My 2 cents more!

'SoS', Ken

In reply to Emma Richardson

Re: Baffling upgrade issue...

by Leon Stringer -
Picture of Core developers Picture of Particularly helpful Moodlers

I just decided to try with a new config file - that allowed the site installer to load (progress!!) - I got to the lang page (the page after entering db details) and it whitescreened there...but just to reiterate that the site connects to the db just fine with 4.1...

Is it worth swapping out the source code and just having the Moodle 4.3 source code, just so we can exclude the 56 additional plugins in one go?

Also is it worth putting in bad credentials for the database (for example, $CFG->dbpass = 'badpassword') and see if there's an error message? If so we'll know it's getting as far as making a connection attempt to the database.

If the database is remote how is $CFG->dbhost specified, IP address or DNS name? Depending on which it is, is it possible to try the other?

Average of ratings: Useful (1)
In reply to Emma Richardson

Re: Baffling upgrade issue...

by Emma Richardson -
Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Plugin developers
I figured it out!! Copied my moodle code folder over to test server which had exact same behavior. Made sure all debugging was turned on for config file and php. Started taking out plugins from local folder one by one to no avail. Finally forced theme to boost which allowed the magical debug message to appear! It was topcoll course format. Took that out and upgrade flew threw - then got all my other plugins upgraded and reinstalled top coll - now I know to pull those down before hand and hopefully will have no issues!
In reply to Emma Richardson

Ri: Re: Baffling upgrade issue...

by Sergio Rabellino -
Picture of Particularly helpful Moodlers Picture of Plugin developers
Sounds good for you, but why on the production server you was unable to catch the same error ?
In reply to Sergio Rabellino

Re: Ri: Re: Baffling upgrade issue...

by Emma Richardson -
Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Plugin developers
Not sure, but when you get lost in rabbit holes, sometimes you forget to do something...I would guess that when I was messing around with config files, maybe I didn't get debugging reenabled or maybe it was a theme issue or a combination of both...think I will try the live site upgrade again in a day or so but feeling much better about it now...
In reply to Emma Richardson

Re: Baffling upgrade issue...

by Ken Task -
Picture of Particularly helpful Moodlers
Congrats! Heck of a way to discover, but maybe a lesson learned.
Before upgrading, force the site to use boost via config.php line and add debugging lines to config.php as well.

Sadly, there is no checker for plugin compatibility.   That's why I use my little bash shell script in checking versions of all installed addons prior to upgrades.   If I see one that shows there is not a version for destination, I'll still try it, but that plugin is the first one I'd hide/remove.

LIke I said before ... a good thing about moodle is that it has plugins .... the bad thing about moodle is the same thing ... it has plugins! sad

But again ... congrats!

'SoS', Ken


In reply to Emma Richardson

Re: Baffling upgrade issue...

by Rick Jerz -
Picture of Particularly helpful Moodlers Picture of Testers
Out of curiosity, Emma, what versions of PHP are one each server?
In reply to Rick Jerz

Re: Baffling upgrade issue...

by Emma Richardson -
Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Plugin developers
8.1
In reply to Emma Richardson

Re: Baffling upgrade issue...

by Rick Jerz -
Picture of Particularly helpful Moodlers Picture of Testers
Last weekend, I upgraded my MAMP from 6.6 to 6.9. It had 8.2 as the default, which was giving Collapsed Topics some trouble. So I backed down to 8.1.13 and my problems went away. So I thought that maybe you had experienced some 8.2 problems. I haven't reported my problems to Gareth yet, but I will soon. I assume that when you say "topcoll" you are meaning Gareth's Collapsed Topics plugin. (Or might it be the other topcoll plugin?)

Another thing that happened to me is that if I made PHP 8.0.8 the topmost choice, MAMP 6.9 crashed on my Mac. 8.0.8 could be the second choice, but not the first.
In reply to Rick Jerz

Re: Baffling upgrade issue...

by Emma Richardson -
Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Plugin developers
It was collapsed topics. I am not using MAMP in this testing scenario - I have WSL set up on a windows box running Debian. I am using that for testing. Really didn't matter though - copying just the moodle code folder over told me that Sergio was right- it was something in my code and so pretty much had to be a plugin. I had been running a 4.3 test on the WSL box prior to copying this over and it ran just fine. Thanks to everyone for all the help though - I just needed to think it all through and then be more thorough in my testing which was easier to do on a test box than my live server...lol...! Thank God for snapshots!