Post-Upgrade Challenges in Moodle 4.4: Dealing with Third-Party Plugin Removal

Post-Upgrade Challenges in Moodle 4.4: Dealing with Third-Party Plugin Removal

by Character Academy Academy -
Number of replies: 37

Im using gpt to help me with a more intelligent summary of what I am experiencing to possibly get to the bottom of this faster. Please dumb down your responses and i can try to make up the difference 🙃

After upgrading Moodle from version 4.3 to 4.4, it looks like the update removed all third-party plugins.

https://drive.google.com/file/d/1pFE7hPbUoSv4_c0d4htAPIUWIFzKQ70Q/view?usp=sharing

Database Update Error

Following the removal of the third-party plugins, I attempted to update the database, which led to another complication. The error message indicated a "DDL sql execution error" related to the qtype_ordering plugin. The error occurred because the qtype_ordering_options table already exists in the database, causing the SQL execution to fail.

https://drive.google.com/file/d/1kBGPUN3I7YEWspEPWKs_56FAXkiWBrcL/view?usp=sharing

Error Details

Here’s a summary of the error message received:

  • Error: ddl sql execution error
  • Debug Info: The error was triggered while trying to create the qtype_ordering_options table, which already exists.
  • Stack Trace: The error trace points to a series of functions within Moodle’s database management system that failed to execute the required SQL commands.

As far as immediate steps taken, i restored everything from a backup before the update. The Moodle Course started to become unstable with so many critical plugins "missing from the disk" 

I dont suppose regular moodle updates like this are intended to remove all third party plugins right? Any suggestions on proposed solutions?

Also wondering if we have a functioning gpt for Moodle updates/processes? If we dont yet, we get someone official to create one here: https://chatgpt.com/gpts

Thanks in advance and looking forward to any suggestions.

Average of ratings: -
In reply to Character Academy Academy

Re: Post-Upgrade Challenges in Moodle 4.4: Dealing with Third-Party Plugin Removal

by Jon Witts -
Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers
How did you carry out your upgrade? Did you follow the steps in Upgrading?

When you download the latest version of the Moodle code it is up to you to also download all of the latest versions of each third party add-on you haver installed (having first checked that they are compatible with the newer version of Moodle). Once, and only once, you have the latest version of the Moodle code and all third party add-ons downloaded should you trigger the upgrade. And most importantly you should be taking a back up of the Moodle Code, Moodle data directory and the database at the same point in time *before* you trigger the upgrade. This way you can get back to where you started should anything go wrong...

HTH
Jon
Average of ratings: Useful (2)
In reply to Jon Witts

Re: Post-Upgrade Challenges in Moodle 4.4: Dealing with Third-Party Plugin Removal

by Character Academy Academy -
Thanks so much for the response here Jon. To answer your questions first - my site is hosted with Godaddy. I accessed the update through a notification on the cpanel which let me know there was an update(see pic). https://drive.google.com/file/d/1Pme89VxLUABL17QXImvkCBcptkFUDnzm/view?usp=drive_link

"Did you follow the steps in Upgrading?" This is a no. My first core moodle upgrade since self hosting(started with MoodleCloud but just recently left), so im still learning the ropes on this one. Great reference point for me, so thank you.

"...Download all of the latest versions of each third party add-on you haver installed (having first checked that they are compatible with the newer version of Moodle)"
Again this is one of my first experiences with the moodle platform I have worked on many more wordpress websites that moodle ones, which i think has a different protocol when it comes to big updates. For some reason, I dont ever recall having to think about plugin much after first download. i just always installed the latest wordpress updates as they came out - and that was it. Since this update just came up in April , I presume It might be good practice just to wait a few months until some of the developers can update their plugins to the new releases. Thats what I think Ill do.

"...Most importantly you should be taking a back up of the Moodle Code, Moodle data directory and the database at the same point in time *before* you trigger the upgrade."

Always always always. I do backups daily. I dont mess around with this in the slightest.😅

I guess the only thing that baffled me on this one is it only removed the third party plugins. None of the built in pluigns were touched. My first thought was "is this standard moodle update?" I had to ask to here to hear someone tell me no its not standard - so im not crazy.

I'm gonna take a look at the updating procedures and follow those more closely. I'll test-run another update to see if I can't produce a different result. WIll keep this thread posted.
Average of ratings: -
In reply to Character Academy Academy

Re: Post-Upgrade Challenges in Moodle 4.4: Dealing with Third-Party Plugin Removal

by Jon Witts -
Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers
So your screenshot which shows there is a Moodle update available, tells me that you have used some kind of one-click installer for Moodle provided (I am assuming) from your hosting provider.

As this one-click installer is not provided by Moodle and the people here really have no knowledge as to what changes have been made to the Moodle code; you may find support for this rather limited... You should probably reach out the provider of the one-click installer for support from them in this matter; or follow a standard Moodle install procedure as documented here: Installing_Moodle

HTH
Jon
Average of ratings: Useful (1)
In reply to Jon Witts

Re: Post-Upgrade Challenges in Moodle 4.4: Dealing with Third-Party Plugin Removal

by Character Academy Academy -
Hey Jon,
I appreciate your followup. I believe you are referencing the "Installatron" that godaddy uses. While I cant comment on how it compares to other installers, or even manual installation methods, i can say it has surely made the transition into development and cpanel easier for me. As far as support goes, they're not very useful. They would just volley me here as its pretty loosely affiliated software with godaddy.

Anyway - I think I got what i waslooking for from another commenter. Basically "keep going". When I saw all the red [missing on disk] after first installation i panicked. Sounds,like i just need to push through and re-install each plugin from backups after the core is in place. I will likely wait until the plugin developers have a chance to update their code and do it in a month.

I knew that moodlecloud wasn't fond third party plugins - as they dont even allow them on their hosted versions. Given how some of these comments have been speaking of third party plugins - It sounds like moodle dev crowd might have a similar feeling towards them. They do break things 😂
Average of ratings: -
In reply to Character Academy Academy

Re: Post-Upgrade Challenges in Moodle 4.4: Dealing with Third-Party Plugin Removal

by Visvanath Ratnaweera -
Picture of Particularly helpful Moodlers Picture of Translators
The upgrade deleted all third-party plug-ins? I haven't dared to 4.4 but have done very many upgrades. It never happened to me.

What is the complication with chatgpt? Have you done what it said and landed here. Then ask it, Why. What we recommend is in the Moodle Docs, in Upgrading, to be specific.
Average of ratings: Useful (1)
In reply to Visvanath Ratnaweera

Re: Post-Upgrade Challenges in Moodle 4.4: Dealing with Third-Party Plugin Removal

by Character Academy Academy -
Thanks for the response Visvanath. Yea I think its best for me to wait a bit longer until there is a bit more time for the plugin developers to test their plugins on the new code too. This is my first moodle site, so Im still learning the procedures when it comes to these updates.

"What is the complication with chatgpt?" No complications. I used chatgpt to generate my initial post for technical specificity. I did insert this into chat gpt and what it suggested was that "The error occurred because the qtype_ordering_options table already exists in the database, causing the SQL execution to fail."

So to my knowledge, the update wanted to create a table , the table was already there. As this is my first time ever "updating" the moodle code through these upgrades, Im a bit thrown for a loop. I think  Im going to try another update and post results.
Average of ratings: -
In reply to Character Academy Academy

Re: Post-Upgrade Challenges in Moodle 4.4: Dealing with Third-Party Plugin Removal

by Visvanath Ratnaweera -
Picture of Particularly helpful Moodlers Picture of Translators
> I used chatgpt to generate my initial post for technical specificity. I did insert this into chat gpt and what it suggested was that [..]

So you asked chatgpt first and fed us what came out? Why didn't you re-feed chatgpt, if it is your oracle? It is rude to ask somebody, a robot for that matter, and then ask others as if it had been what you were thinking.

> So to my knowledge, the update wanted to
No, it was not *your* knowledge and I'm out of this discussion.
sad

To the moderators: If is time you take a policy decision on this nuisance. The previous major nuisance was not too long ago: See Pursuing AI bots on moodle.org forums...
Average of ratings: -
In reply to Character Academy Academy

Re: Post-Upgrade Challenges in Moodle 4.4: Dealing with Third-Party Plugin Removal

by Ken Task -
Picture of Particularly helpful Moodlers

+1 to Jon's and Mr. V's response:

https://docs.moodle.org/404/en/Upgrading

wasn't followed.

And 2 cents more ... site is behind CloudFlare (CF) ... which protects site from inbound maliciousness ... your attempt to upgrade via browser could have been interpreted by CF as just that.

Best, me thinks, to use command line on server and git for upgrades because that doesn't go through CF.   Code stays in place and new core code is literally pulled into existing moodle code ... no moving of config.php or all those plugins you have installed.

Comments ... for what they are worth ...

I, for one, am not gonna work on your ChatGPT thang to solve your issues.  Not for free anyway!   Kinda brash for you to ask, isn't it?

Besides ... your brain will atrophy ... don't want to be a party to that! smile

'SoS', Ken

Average of ratings: Useful (1)
In reply to Ken Task

Re: Post-Upgrade Challenges in Moodle 4.4: Dealing with Third-Party Plugin Removal

by Character Academy Academy -
Thanks for your response to the post Ken.
"I, for one, am not gonna work on your ChatGPT thang to solve your issues. Not for free anyway! Kinda brash for you to ask, isn't it?"
My apologies it if came across a request for free services. Not my intent at all. Just need a tad bit of guidance as this is my first moodle site and first moodle update. Lots of "firsts" on this one so i Just want to know if Im facing the right direction 😅

Im actually trying to perform the update on a Godaddy hosted shared server(see pic) https://drive.google.com/file/d/1Pme89VxLUABL17QXImvkCBcptkFUDnzm/view?usp=drive_link

I get notifications of updates on the cpanel and thats where I initiated this one. I'm going to read through the upgrading manual a bit more closely to see if i cant find a procedural tip that might help alleviate my problems. Hopefully I'll have positive news to post in my next update. Cheers!
Average of ratings: -
In reply to Character Academy Academy

Re: Post-Upgrade Challenges in Moodle 4.4: Dealing with Third-Party Plugin Removal

by Ken Task -
Picture of Particularly helpful Moodlers

In your screen shot ... where it list servers that you supposedly control and have setup in GoDaddy ... the first one ... doesn't seem to resolve in DNS - site not found.

The third site listed there is a moodle.org/course/   I hope that's a link and not something you've attempted to setup.   That goes to forums here.

Comments:

your experiences with WordPress may/may not apply with a Moodle.

'shared hosting' ... one doesn't have much control over areas of the server that you will need if your 5 commercial course offerings begin to attract customers

Afraid you are gonna have to up your game on asking questions and posting in a technical sense ... so far, and no offense meant, we are kinda getting 'the thingy broke!   Help me fix the thingy!' - pretty close to that.

So in your moodle site, learn how to turn on debugging (to the max), do whatever caused the error, then come back here and share the text of the debug output - removing any sensitive info ... like logins/passwords/etc.

Let the debug info talk tech!

You are beginning a journey!   Hopefully, you will be able to stay above the water line of the iceburg that is called Moodle! smile [but you will have to go below that water line for some things ... example: cron job setup is done in your cPanel ... not in Moodle]

'SoS', Ken

Average of ratings: -
In reply to Character Academy Academy

Re: Post-Upgrade Challenges in Moodle 4.4: Dealing with Third-Party Plugin Removal

by Matthias Giger -
Picture of Particularly helpful Moodlers
When you upgrade to a higher Moodle version, you need to remove the old code, otherwise you will get an error message along the way that your instance uses "mixed code" or something similar. That is way, the whole /moodle folder will be deleted. Along with that every third party plugin is also erased.

Once you have the new Moodle code in place and start the update process manually, you will get a notice that 3rd party plugins are missing from disk. That's the time you reinstall this plugins. You can either do that by copying all the folders from your backup into the new installation or try to do this with git.

Just reload the update page (don't press any buttons yet), until all the missing disk messages have disappeard. Now, you are really ready to go through the update process. There is one thing to do, in case you use special course formats, you have to additionally install them, as for some reasons there are not showed in the upgrade page.

Have you put everything in place, you go throught the update process. Normally, this works with most plugins, but a few of them might cause problems. That is the reason why some people don't install additional plugins.

As far as Moodle 4.4. is concerned, there were very few problems, I experienced. One of the exceptions was the preinstalled ordering question type. In my case, it needed some tampering with the SQL database. This hickups are the reason why you test an update before you apply it to a productive environment.

And one last thing, never use an auto-installer, always do you updates either by hand or with git. Auto-installers will sooner or later lead to a result which makes you cry.
Average of ratings: Useful (2)
In reply to Matthias Giger

Re: Post-Upgrade Challenges in Moodle 4.4: Dealing with Third-Party Plugin Removal

by Visvanath Ratnaweera -
Picture of Particularly helpful Moodlers Picture of Translators
Matthias, that is excellent!

Don't tell me, you've found it in chatgpt. Or may be tomorrow, wait till chatgpt digests that and regurgle on to us. Well, there is Upgrading, why doesn't chatgpt guide the seeker there?
smile
Average of ratings: -
In reply to Matthias Giger

Re: Post-Upgrade Challenges in Moodle 4.4: Dealing with Third-Party Plugin Removal

by Character Academy Academy -

Hi Matthias,

Thank you for sharing your insights here. This is one of the most helpful comments on this thread so far. It sounds like your suggesting that Moodle does automatically delete the third party plugins due to "mixed code" concerns.  And I can understand that if thats their protocol. It definitely adds an additional layer of work for anyone wanting to use non CORE Moodle plugins , but it is what it is 😅

"That's the time you reinstall this plugins. You can either do that by copying all the folders from your backup into the new installation or try to do this with git."

I can do this. My initial reaction to all the pllugins being deleted was I did something wrong. It sounds like you are suggesting i "proceed ahead" once the core moodle is in place. Thats fine with me

 "a few of them might cause problems. That is the reason why some people don't install additional plugins.

I totally get this. This is actually why I might wait at least a month or so longer to make the update. I would like to give the developers a chance to react and respond with their work. 

"never use an auto-installer, always do you updates either by hand or with git. Auto-installers will sooner or later lead to a result which makes you cry.

Haha I will take this into consideration. I am taking my programs development one week at a time, trying to learn as much as i can so that i can take responsibility for as much as I can when it comes to management. Development is just one of the many hats im wearing at the moment and learning git is on the top of my list. 

Thank you again for this helpful response Matthias. I'd give it top comment if i could ⭐⭐⭐⭐⭐

Average of ratings: -
In reply to Character Academy Academy

Re: Post-Upgrade Challenges in Moodle 4.4: Dealing with Third-Party Plugin Removal

by Leon Stringer -
Picture of Core developers Picture of Particularly helpful Moodlers

It sounds like your suggesting that Moodle does automatically delete the third party plugins due to "mixed code" concerns.

Moodle does not automatically delete third party plugins during the upgrade, nor at any other time (unless you click the corresponding delete link on the plugins overview page).

I think Matthias is suggesting you should remove the third party plugin source code as part of your upgrade process to identify which third party plugins you need to obtain. Personally I would check third party plugin compatibility ahead of the upgrade, then prepare the target Moodle version's source code with any necessary updated third party plugins.

Obtaining updates for third party plugins which are compatible with your target Moodle version is covered in the upgrading instructions Jon and Ken linked to.

Average of ratings: Useful (1)
In reply to Character Academy Academy

Re: Post-Upgrade Challenges in Moodle 4.4: Dealing with Third-Party Plugin Removal

by Ken Task -
Picture of Particularly helpful Moodlers

Am gonna encourage you to pursue command line and git and moosh for your site.

Here's why ...

If one waits for plugin developers to have compatible versions for 4.4, when and how will you know it's OK to update?

How many additional plugins do you have?

Login as admin, go to Plugins, Plugins Overview, Additional Plugins.
There will see a list.
Left Column the long name of the plugin and right underneath that long name
a short name.
Example: Collapsed Topics (long name)
right under the short name: format_topcoll

Can you provide a list of those short names here?

Or, you can go to the Moodle plugins site:
https://moodle.org/plugins/?q=
and search of them using the short name

Clicking into what that search finds for the example Collapse Topics Format
One can see info:

https://moodle.org/plugins/format_collapsibletopics

and comments:

"Warning: since sections collapsing behaviour is now integrated into Moodle core, this plugin will not be maintained for Moodle 4.x The collapsible topics format is a clone of the core topics format that adds collapsing behaviour to sections in order to reduce page size.

Warning: since sections collapsing behaviour is now integrated into Moodle core, this plugin will not be maintained for Moodle 4.x

The collapsible topics format is a clone of the core topics format that adds collapsing behaviour to sections in order to reduce page size. A plugin setting allow to retain topics collapse state on a per user basis for the current session or across sessions. This plugin is an alternative to Gareth J. Barnard's format_topcoll plugin (see references below) The purpose is to offer a simple collapsing behaviour for sections in order to reduce course home page size without all the display options that Gareth's plugin offers."

Now you have a plan/warning you can then be forewarned or take a different direction.

There is another way - more efficient but it involves command line and moosh.
Moosh is the Swiss Army Knife utility for Moodle - command line only.

https://moodle.org/plugins/view.php?id=522

And using a moosh command

moosh -n plugin-list |grep format_topcoll

to check for the same plugin above one gets info needed

format_topcoll,1.9,2.0,2.1,2.2,2.3,2.4,2.5,2.6,2.7,2.8,2.9,3.0,3.1,3.10,3.11,3.2,3.3,3.4,3.5,3.6,3.7,3.8,3.9,4.0,4.1,4.2,4.3,4.4,https://moodle.org/plugins/download.php/31753/format_topcoll_moodle44_2024032800.zip

I can see there is a version for format_topcoll plugin through 4.4
And the zip is the plugin for moodle 4.4.

Anyhoo, Moodle is an iceburg (compared to WordPress/Joomla/etc) and there are times where one needs to get below the water line (command line) to be more efficient and less prone to human error.

It's a journey.

Will say this ... the time you spend learning command line ... git, moosh and the admin/cli/ scripts in core code ... will be well worth it.

You can spend more time developing content and less time maintaining/updating/upgrading.

'SoS', Ken

Average of ratings: Useful (1)
In reply to Ken Task

Re: Post-Upgrade Challenges in Moodle 4.4: Dealing with Third-Party Plugin Removal

by Gareth J Barnard -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Plugin developers

Dear Ken et al,

To optimise your reply example, you can take the 'short name', which is the 'Frankenstyle' and append to 'https://moodle.org/plugins/' to find the plugins entry in the plugins database, i.e. https://moodle.org/plugins/format_topcoll

Upgrading wise I strongly recommend https://docs.moodle.org/404/en/Git_for_Administrators - and I don't use submodules for contributed plugins, rather they have their own .git folder within their frankenstyle location, i.e. Collapsed Topics lives in 'course/format/topcoll'.  That way when you've done your preparation and know that there is a release for all the additional plugins you have, then you can update from the respective branch.

Another approach is to leave your production alone.  Create a fresh test server installation, then install by whatever means (manually or Git) everything that you want and test it completely to be happy that it works.  Once that's done.  Do a complete backup of production, put into maintenance mode, remove all of the code files bar the main 'config.php' - leaving 'moodledata' and your database alone.  Then copy all (bar the main config.php file) files and folders from the test install code folder to the production, go back to the UI and update as normal etc.  If this process is scary, then practice with two test installs of a lower and higher version where you migrate one from the other.

And if you're stuck, don't ask ChatGPT as its just guessing, post on a forum here for a human reply, specifically the one aimed at the plugin you have an issue with.  I regularly watch both the 'Themes' and 'Courses and course formats' forums several times a day, seven days a week.

Gareth

Average of ratings: Useful (1)
In reply to Gareth J Barnard

Re: Post-Upgrade Challenges in Moodle 4.4: Dealing with Third-Party Plugin Removal

by Rick Jerz -
Picture of Particularly helpful Moodlers Picture of Testers
I have been doing something like you suggested about leaving the production alone.

First, I do everything in my experimental MAMP.
1) I check my current Moodle to see if any additional plugins (four) need to be updated. If so, I manually download the new plugin, unzip it, and replace the production Moodle plugin. I do this by deleting the folder that contains the plugin, the move the new plugin into its place, and update the production Moodle. At this point, I know that my production Moodle's plugins are up to date.
2) I download the new Moodle into a temporary folder and unzip it.
3) Then I run a shell script that does the update by:
3a) Backup moodle, moodledata folder, and database.
3b) Copy the current production Moodle plugins into this newly unzipped Moodle.
3c) Copy any "code" modifications into this newly unzipped Moodle.
3d) Rename the production Moodle to moodlex.
3e) Move the new Moodle in the temporary folder to the production location.
4) Log into Moodle to complete the upgrade.

This update in MAMP is very quiz, under 5 minutes.

Once I verify that the new Moodle looks good in MAMP, I repeat this process on my server's production Moodle.

Someday, I might use GIT, but so far, I am satisfied with my manual approach.

Also, someday, I will make a video showing my method.
Average of ratings: -
In reply to Rick Jerz

Third-party plug-ins and Git (was Re: Post-Upgrade Challenges..)

by Visvanath Ratnaweera -
Picture of Particularly helpful Moodlers Picture of Translators
To all

On third party plug-ins and Git: 

I am learning things in the discussion Versioning and Deprecation changes, in the sub-thread starting here. In short, Git may (still) not be the best way to install plug-ins. Please get the details from there, I don't know more.

Average of ratings: -
In reply to Visvanath Ratnaweera

Re: Third-party plug-ins and Git (was Re: Post-Upgrade Challenges..)

by Ken Task -
Picture of Particularly helpful Moodlers

Git for core only ... plugins are another matter.   I let moodle sort those out ... and it does a good job of doing that - I won't go so far as to say it's a 'no-brainer' - but darn near!  smile

I have 4 servers on different networks with 4.1.x -> 4.4 versions of Moodle and will be doing point release updates today.  It will take about 2 hours ... and those all use a command line script called 'up' that will put site in maintance mode, make a site backup first before the update, check the environment and pause at that point, if all OK, then git will acquire the point released code (I move nothing) for core, and perform the upgrade via cli, then purges the caches, takes site out of maintenance mode for me to check.

BTW, I've already check compat of the plugins on all those servers via moosh.

No, we all don't have to do things the same way ... we all choose our method of 'madness'!   I just find my method to be simple and accurate - less prone to human error and quick compared to the 'other methods of madness'! smile

My 2 'sense' - nope spelled it right!

'SoS', Ken

Average of ratings: Useful (1)
In reply to Rick Jerz

Re: Post-Upgrade Challenges in Moodle 4.4: Dealing with Third-Party Plugin Removal

by Ken Task -
Picture of Particularly helpful Moodlers

The following is not about winning or loosing ... but ...

@Rick a question for you ...

When moodle releases a 'point' update to your current version of Moodle,
do you acquire them via your process?

Do you ignore the 'Update notifications' telling you there is a newer version
of Moodle available ... and the recommendation of and I quote:
"It is strongly recommended that you update your site to the latest version to obtain all recent security and bug fixes."

Moodle 4.3 is now 4.3.4+, Moodle 4.4 is now 4.4+

There is an advantage in using Git for core ... not the plugins.

'SoS', Ken

Average of ratings: Useful (1)
In reply to Ken Task

Re: Post-Upgrade Challenges in Moodle 4.4: Dealing with Third-Party Plugin Removal

by Rick Jerz -
Picture of Particularly helpful Moodlers Picture of Testers

Q1: Yes, I do the same thing for point releases.  I do take a little more time when I first upgrade MAMP to look more closely to see if anything has significantly changed before applying the point release to my production server Moodle version.

Q2: I do ignore the "Update notifications."  Perhaps I ignore them because I know that on about every Friday/Saturday, a new release of Moodle will happen, and then every Friday/Saturday I upgrade anyway.  So maybe I am not really ignoring them, I just never look for them.  

Average of ratings: -
In reply to Rick Jerz

Re: Post-Upgrade Challenges in Moodle 4.4: Dealing with Third-Party Plugin Removal

by Ken Task -
Picture of Particularly helpful Moodlers

I see ... a bit more ...

https://moodledev.io/general/releases
General release calendar
These are the target dates for releases. These dates may vary slightly due to unforeseen circumstances.
Release type    Frequency    Months    Timing
Major (eg. 3.x)    6 monthly    April, and October    Weeks 17, and 43 of the year
Minor (Point) (eg. 3.x.y)    2 monthly    February, April, June, August, October, December    Usually every 8 weeks

Doesn't happen often but ... 
"These dates may vary slightly due to unforeseen circumstances"
such as in a 0 day flaw ... that might or might not be 'in the wild' (actively exploited).

That's just moodle ... then there is your operating system as well.
How often does your GoDaddy server inform you of OS updates?

The 'black hats' are always a little ahead of the 'white hats'!

'SoS', Ken

Average of ratings: -
In reply to Ken Task

Re: Post-Upgrade Challenges in Moodle 4.4: Dealing with Third-Party Plugin Removal

by Rick Jerz -
Picture of Particularly helpful Moodlers Picture of Testers
My brain is programmed to look at Moodle.org's "Downloads," "Moodle 4.4+ (right now), and then the latest stable version. If this file is less than around 7 days old, I download it and go through my process. Every time I upgrade, I make an entry in my "Moodle Notes" document like so: "Moodle 4.4+ (Build: 20240524) on 5/25/24." Also, every time before upgrading, I make a complete backup of my Moodle.

I look at that "General release calendar" to get a sense of what major updates might be on their way. I don't look at Site Admin, Notifications very often.

My GoDaddy VPS doesn't email information about OS updates. I tend to log into WHM about once every 3 to 4 weeks. While logged in, I often look to see if there are updates. If so, I (blindly) do these then I do a "graceful reboot" of my server. There have been times when I have gone months without touching my GoDaddy VPS. Before I do the graceful reboot, I do run "top" in terminal and take a screenshot, just for archival purposes.

My Moodle server backup script keeps a moving backup of the last 7 days. Since I am limited with server space (sure, more $$'s for more space on GoDaddy), I have a shell script on my Mac that every day moves the backups from my server to a big external hard disk on my Mac. My "big external drive is 8TB, which is much more affordable than 8TB on a VPS." I have a cron job on my Mac that runs every evening. I have three folders: daily, weekly, and monthly. I keep the last 10 days, the last 8 weeks, and the last 6 months of backups (that is, one per week and one per month.) (My backup method is actually more complex, but what I have said is good enough for now.)
 
I do not do "course backups." I back up my entire Moodle once each day.  This is my comfort zone, somewhere between a black and white hat.
Average of ratings: -
In reply to Rick Jerz

Re: Post-Upgrade Challenges in Moodle 4.4: Dealing with Third-Party Plugin Removal

by Ken Task -
Picture of Particularly helpful Moodlers

Wow!  Full disclosure! smile

I could see where you would admin a moodle differently than others.   Fits what you do.

But for a site with 400+ courses with over 11,000 users - teachers/students with a teacher as a volunteer moodle admin and an IT department of 1 person to cover all the internal server sevices an ISD needs - different needs - different solutions.

With that site, we are facing a migration from CentOS 7 to a Rocky 9 this summer.

In preparation for that am trying to make the current site as lean and mean as I can.

Corps have different ideas and needs.

I, for one, wouldn't host anywhere that doesn't offer detachable data drives ... cheap - $2.12 a month not bad for 100 Gig data device.   Detachable data drives can be attached to any server you have on their network.   Even better, a mounted Google Bucket - 1 Pentobyte - I've never come close to filling that up in many years.

But have temp mounted that bucket on the ISD site to archive 2 sites - production and a sandbox + all courses - 2 ways - with students and without.

WHM/cPanel setup are GoDaddy's caddie isn't it?

Did you ever notice you are your own customer with that setup?

And that cPanel does have git ... one of these days you are gonna give that a shot ... just for core!

Thanks for sharing!

Now if OP hasn't been overwhelmed with info he will jump back in here and share with us what he decided and if further assistance is needed! smile

'SoS', Ken

Average of ratings: -
In reply to Ken Task

Re: Post-Upgrade Challenges in Moodle 4.4: Dealing with Third-Party Plugin Removal

by Rick Jerz -
Picture of Particularly helpful Moodlers Picture of Testers

GoDaddy offers both a "Plesk" and "cPanel" control panels.  The cPanel is $20/month, Plesk $30/month.  I always buy the cPanel option because I have had better luck with it.  I can't quite figure out the "cPanel" name because cPanel includes WHM.  WHM is the top-level VPS management system.  When you create accounts and want to manage an account, you then use cPanel for each account.  Someday I will figure this out.  I assume that I am not unique and that this WHM/cPanel combo is what everyone gets.

Some utilities exist in both products, as you would probably expect.  Like "Terminal."  However, when you "Terminal" at the WHM-level, you are "root."  At the cPanel-level, you are the account owner's name.  At the cPanel-level, you can "su" to root, if needed.

Yes, GIT is included in cPanel (as expected), but not in WHM.

Yes, I know we have gone off-course from the original post, but it's a lot of information for the OP.

Just as a note, in my method I do take care of plugins, somewhat indirectly, perhaps.  But another thing that I take care of are some of those "nasty" code modifications that we don't recommend people doing.

I would post my script, but I need to clean it up a little bit for humans to be able to read it. 💁‍♂️

Average of ratings: Useful (1)
In reply to Rick Jerz

Re: Post-Upgrade Challenges in Moodle 4.4: Dealing with Third-Party Plugin Removal

by Ken Task -
Picture of Particularly helpful Moodlers

WHM/cPanel or Plesk setup allows you to be a reseller ... if you wanted to start your own moodle hosting offering for others to use on a commercial scale you could.  Could call it ricksmoodlehosting.com - hey, that domain name isn't registered ... might wanna grab that up and park it on GD for when you retire!

Wow! GD charges for cPanel?  About $250.00 a year!

Is that an either/or deal or could you opt out of either?

Reason asked - other open source panels exist.   Webmin for example - which would be like cPanel for the customer and VirtualMin - which would be like WHM.

Webmin does things that cPanel/Plesk can not do ... for one, 'Custom Commands' which can point to your shell scripts and execute them ... can even pass parameters to those custom commands as well as edit those scripts.

Nice thing about Webmin/VirtualMin - Free - updates themselves.  Have often thought moodle coders could take a page from how Webmin updates itself cause the process is similar to the 'old way' of updating/upgrading a moodle.

When I get lazy I can login to webmin, go to Custom Commands, and click a button I've created that runs my 'up' script in a terminal window which uses git.  Matter of fact, I have created a customer account for use in webmin then restricted that customer account to be able to use certain Webmin modules - but can't change configs.   They could then click the Custom Command 'Update Moodle' button all by themselves.

'SoS', Ken

Average of ratings: -
In reply to Ken Task

Re: Post-Upgrade Challenges in Moodle 4.4: Dealing with Third-Party Plugin Removal

by Visvanath Ratnaweera -
Picture of Particularly helpful Moodlers Picture of Translators
> Wow! GD charges for cPanel? About $250.00 a year!

Goodness! Now I know how much the CLI saves me, at a minimum.
big grin
Average of ratings: -
In reply to Visvanath Ratnaweera

Re: Post-Upgrade Challenges in Moodle 4.4: Dealing with Third-Party Plugin Removal

by Rick Jerz -
Picture of Particularly helpful Moodlers Picture of Testers
Yep, GD charges for cPanel. But when I search for "is cPanel free" I find "You must purchase a cPanel license for each cPanel & WHM server that you wish to run." So then I go to the cPanel store and see that the least expensive license is $17.49/month. But this is for one account. As Ken somewhat points out, someone can buy a VPS and then resell smaller accounts to others. WHM is what the VPS owner uses to manage the server, and cPanel is what each account holder can use to manage their website. I see on the cPanel store that a 5-account cPanel can be purchased for $30/month, and a 30-account version for $43/month. So I don't know, is the $20/month GD charges reasonable? I could resell as many accounts as I want.

Yes, if you know how to manage a Linux server, you wouldn't need cPanel. Perhaps someday I will not need it.

Ken, I do own several domains, and my "production" moodle is located on one. I do wonder what I might do in the future... not when I retire, but when I "die." This question bothers me a lot. 💀
 
Yes, there are some free "panels" but then I would have to figure out how to install them on my VPS.  But Ken, you have me thinking.
 
Regarding the OP's original question, I do not use WHM/cPanel to manage third-party plugins. My two tools are "Terminal," which I run from my Mac, and an FTP client called "Forklift" that I also run from my Mac.  My Mac is also not free! 😀💰💰💰💰💰.  Forklift isn't free, but by golly, Terminal is free (courtesy of Apple. Thanks.)!
 
 
Average of ratings: Useful (1)
In reply to Rick Jerz

Re: Post-Upgrade Challenges in Moodle 4.4: Dealing with Third-Party Plugin Removal

by Ken Task -
Picture of Particularly helpful Moodlers

Wow!  You do all that for a point release?

Here's mine for a 4.4+ sandbox on a Rocky 9 server:

No build local and FTP up to server.
No moving parts.

Just ssh into server: 
cd /path/to/codedirectory/
./up [ENTER]

Then watch for the prompts and hit enter - could cancel the script at those prompts.

Script with some minor obscures []

echo 'Currently: 4.4+ (Build: 20240516)'
tar -cvf /mnt/data/backup/m44sb/msb44+-$(date +%Y%m%d%-H%M%S).tar ../m44sb;
tar -cvf /mnt/data/backup/m44sb/msb44+-data-min-$(date +%Y%m%d%-H%M%S).tar /var/www/m44sbdata/filedir;
mysqldump -u root -p'[password]' m44sb > /mnt/data/backup/m44sb/msb44+-db-$(date +%Y%m%d%-H%M%S).sql;
ls -l /mnt/data/backup/m44sb/;
echo 'Paused ...';
read $keypress;
git branch -vvv;
echo 'Paused ...';
read $keypress;
php admin/cli/cron.php;
echo 'running checks ...';
php admin/cli/checks.php
echo 'Checks OK?';
read $keypress;
echo 'Ok, updating ...';
php admin/cli/maintenance.php --enable;
git pull;
php admin/cli/upgrade.php --non-interactive;
php admin/cli/maintenance.php --disable;
chown apache:apache * -R;
chown apache:apache /var/www//m44sbdata/ -R;
chown root:root up bu bugb upgrade;
chmod go-rwx .git .github .gitattributes .gitignore -R;
php admin/cli/purge_caches.php;
fgrep '$release' version.php


end result:

Maintenance mode has been disabled and the site is running normally again
$release  = '4.4+ (Build: 20240524)'; // Human-friendly version name

Done!

I do plugin updates via admin interface - saves time and code is good at sorting out what version.

'Spirit of Sharing', Ken

Average of ratings: -
In reply to Ken Task

Re: Post-Upgrade Challenges in Moodle 4.4: Dealing with Third-Party Plugin Removal

by Rick Jerz -
Picture of Particularly helpful Moodlers Picture of Testers
Thanks much for sharing. I always learn something from you.

In my spirit of sharing, I am attaching my upgrade script. Perhaps those who are following our conversations will benefit from these.  It is not the most elegant script, but I try to include documentation (comments) in it to remember what it does.  Of course, I am always interested in suggestions.  For example, Ken, a while ago, you suggested putting Moodle credentials in a separate file and reading them with my backup script.  I did this, so you won't see my Moodle credentials in this script.  I do my upgrades in the morning when no student is logged in, so I don't put my Moodle into maintenance mode, but I think I will integrate your code because it won't hurt anything.

I am only managing one production, Moodle, and one experimental, Moodle. My method takes under 10 minutes each week. Your method is definitely faster.
 
I also do plugin updates manually.  Some plugins will email me when they have been updated.  I am unsure if all do.
Average of ratings: Useful (1)
In reply to Rick Jerz

Re: Post-Upgrade Challenges in Moodle 4.4: Dealing with Third-Party Plugin Removal

by Visvanath Ratnaweera -
Picture of Particularly helpful Moodlers Picture of Translators
Ken and Rick, very useful collection of information for those who run small Moodle instances on shared hosting with and without shell access. My only concern is that since https://moodle.org/mod/forum/discuss.php?d=458490#p1841907 we are not talking Post-Upgrade Challenges in Moodle 4.4: Dealing with Third-Party Plugin Removal.
Average of ratings: -
In reply to Rick Jerz

Re: Post-Upgrade Challenges in Moodle 4.4: Dealing with Third-Party Plugin Removal

by Ken Task -
Picture of Particularly helpful Moodlers

Thanks for sharing back.   Wow!  Well commented - am too lazy to do that! :|

I do have a mdlvars file which defines variables used in up, update, bu scripts.

Much like you have done with your .sh.

Does make the script transportable to any Linux system.

Suggestion: at the very end of your .sh script, do the upgrade via command line:

php codedir/admin/cli/upgrade.php --non-interactive

and afterwards - purge caches via the script to do so in admin/cli/

Moodle will start to rebuild those caches even before you can login again (cron job/task).

Once an upgrade/update has been successful, do a site backup ... and only after you are sure the upgrade/update is working as it should, go back and remove old backups cept the one made just previous to the update/upgrade.

That saved my bacon recently ... the DropBox issue.

'SoS', Ken

Average of ratings: -
In reply to Ken Task

Re: Post-Upgrade Challenges in Moodle 4.4: Dealing with Third-Party Plugin Removal

by Rick Jerz -
Picture of Particularly helpful Moodlers Picture of Testers
Okay, I will add your suggestions. Thanks.
Average of ratings: -
In reply to Rick Jerz

Re: Post-Upgrade Challenges in Moodle 4.4: Dealing with Third-Party Plugin Removal

by Character Academy Academy -
Hi Rick,
Thanks so much for commenting. Firstly, just want to say I'm really glad to see another godaddy user here. Its reassuring that I'm not the only one still using the platform (they don't have the most sterling reputation in all groups especially when it comes to hosting).

Besides that point...

Some of what Im getting from this thread was to be expected - I'm reading that many of us here are involved with moodle from vastly diferent starting points ( such asKen who seems to be "a volunteer moodle admin and an IT department of 1 person to cover all the internal server sevices an ISD needs for a 400+ courses with over 11,000 user implementation) Way different from my use case. lol I'm actually just a teacher with some developing experience, not a sys admin by ANY stretch of the title.

Anyway, you are absolutely correct! there is a lot of great information in this thread. Im trying my best to digest all that I can without getting overwhelmed 🤪(but i may be too late). Just how crazy i am and how wrong I'm doing things of course depends on who you ask lol Unfortunately Im on a shared server right now which makes a difference in how much control i have over my system. It sounds like your on a VPS - which is the goal eventually, but not where Im at right now. So that dictates what tools I'll be using right now and what i can do.

One of the big take-aways I getting from this thread is that I dont really have much in place by way of processes / protocols. So Im trying to pick up on SIMPLE processes that will keep me moving forward while I download and eventually learn all of the sys admin ninja tricks everyone has to share here lol

As for right now, i'm planning on just updating my looong list of things to do (Git is on the agenda, but not useful without root, mamp is something i need to put on the list, moosh was mentioned and sounds helpful ), backing up my entire site(plugins and all), installing the core update, and then reinstalli each plugin from the backups one by one. Hopefully that will get me over the finish line on this one and may be in a place that I can pencil in a process or two in preperation for the next update in Nov. Its been a little over a month since the release became public so I figured some developers have at least looked at the release 🤣 Feel free to plug any programs i missed or (simple) processes I can take as i cross this finish line lol
Average of ratings: -
In reply to Character Academy Academy

Re: Post-Upgrade Challenges in Moodle 4.4: Dealing with Third-Party Plugin Removal

by Rick Jerz -
Picture of Particularly helpful Moodlers Picture of Testers
Yep, I've been using GoDaddy since 2006. However, if someone prefers a different hosting company, so be it. I don't work for GoDaddy and don't own their stock.

I am also just a teacher and not a "sys admin." I love learning from folks like Ken, Howard, Visvanath, et. al. I rely on a solid server platform and "cPanel" tools. I made the two videos below, at Ken's request. They may be helpful. But I don't have a video about "migrating" yet.

Yep, can't do as much from a "hosted" server. I learned this many years ago when I switched to a VPS. I promote buying a one-month VPS for practice, like buying a book.



More discussion, and videos: https://moodle.org/mod/forum/discuss.php?d=401983#p1621924

Any more about a sandbox. This is a good place for you to learn.



Average of ratings: -
In reply to Ken Task

Re: Post-Upgrade Challenges in Moodle 4.4: Dealing with Third-Party Plugin Removal

by Character Academy Academy -

Hi Ken,

Thanks for all of the suggestions here. This post has generated so many great conversations and helpful tips, and I'm working to take into account as many as i can muster. 😅

I've looked into git for version control, however since I am on a shared server, that gives me limited access to root directory and therefore git functionality is severely limited. That means if im going to practice with git, its gotta be external to my situation which just further complicates learning that. I do intend to upgrade to a VPS eventually, but not right now Im still going to try my best to learn what I need to know as I go. It sounds like Moosh is another great suggestion too. I do intend to stay with moodle for the long-term so this sounds like a great resource.

"Anyhoo, Moodle is an iceburg (compared to WordPress/Joomla/etc)"

I too have gotten this impression😂 After spending a week or so creating a few database queries using https://www.examulator.com/er/4.0/ I see that the schema shows that moodle has a lot more folds that it doesnt show at a quick glance. 

"there are times where one needs to get below the water line (command line) to be more efficient and less prone to human error." 
Not gonna lie, I shoud know command line by now. Frankly it's embarrasing that I dont but it is what it is. I always operate under the assumption that I cant swim lol need floaties asap, im not ashamed to admit that at least 🤣

Average of ratings: -
In reply to Ken Task

Re: Post-Upgrade Challenges in Moodle 4.4: Dealing with Third-Party Plugin Removal

by Visvanath Ratnaweera -
Picture of Particularly helpful Moodlers Picture of Translators
> there are times where one needs to get below the water line (command line) to be more efficient and less prone to human error.

And some love it and stay there forever. See what happened to Elsie, Lacie, and Tillie;

===
`Once upon a time there were three little sisters,' the Dormouse began in a great hurry; `and their names were Elsie, Lacie, and Tillie; and they lived at the bottom of a well--'

`What did they live on?' said Alice, who always took a great interest in questions of eating and drinking.

`They lived on treacle,' said the Dormouse, after thinking a minute or two.

`They couldn't have done that, you know,' Alice gently remarked; `they'd have been ill.'

`So they were,' said the Dormouse; `very ill.'

Alice tried to fancy to herself what such an extraordinary ways of living would be like, but it puzzled her too much, so she went on: `But why did they live at the bottom of a well?'

=== https://www.cs.cmu.edu/~rgs/alice-VII.html

Average of ratings: -