changing to git on production stack + manually installed plugins

changing to git on production stack + manually installed plugins

by Matt T -
Number of replies: 9

Right now our production stack contains moodle installed manually including (many) plugins. For the purposes of this post "manually" means "not by git". To simplify security updates I'd like to change it to use Git.

 I've read over the guides but still slighly confused. They seem to discuss adding extensions by git but not the situation when your extensions are installed by zip upload.

My plan right now is to git checkout the latest moodle branch into /var/moodle, copy config.php and extension directories and then add those directories to a .gitignore file. Then presumably git pull wouldn't be confused that those directories don't exist on the upstream repo? Finally set appropriate permissions and point apache to the git-ified directory (so there's no downtime).

Then should I need to add new extensions presumably they will need to be added to .gitignore as well

Does this sound right or is there a better way to do this? 

Average of ratings: -
In reply to Matt T

Re: changing to git on production stack + manually installed plugins

by Ken Task -
Picture of Particularly helpful Moodlers

In the past, experts like Howard (moderator of some forums), had suggested 'side loading' via git the same version as production code.   The copy with preserve the hidden .git diretory and .gitornore files into production directory.   Less moving of files/folders ... some of which are easy to miss

Backup code directory and DB before trying this!  (you've been warned)

Are the plugins/addons compatible with the version to which you will upgrade?   Check those.

Addons that were not installed by git won't be known to git ... git only knows about core code.  Have you hacked any core code?  Even just one line or a few characters?   If so, git might recognize that.

What is current version of code?   in /var/www/html/ (as per example below),

fgrep '$release' version.php will tell you.

Example:

Production site is in /var/www/html/ and is version 2.9.x

at /var/www/ use git commands to install the highest 2.9.x in a directory by another name at the same level as /var/www/html/

git clone git://git.moodle.org/moodle.git htmlgit (IF git does not work, stop ... check firewall ports, boundary firewall ports, etc.

That creates an htmlgit directory with git acquired dev code.

change into htmlgit and issue

git branch --track MOODLE_29_STABLE origin/MOODLE_29_STABLE

then

git checkout MOODLE_29_STABLE

To check what branch htmlgit is now pointed to: git branch -a

Should show a 29 stable as branch - highest 2.9.

Now copy the hidden .git directory and .gitignore to production directory.

in htmlgit

cp -rp .git /var/www/html/

cp -rp .gitinore /var/www/html/

cd /var/www/html/

Issue git branch -a

Does it show the highest 2.9?

The, just test git:

git pull

Any new code?

If git acquires new code ... check version.php file for what it acquired.

May as well finish the upgrade while at command line

php admin/cli/maintenance.php --enable

php admin/cli/upgrade.php --non-interactive

chown apache:apache * -R (for Ubuntu www-data:www-data ... or other ?)

Check version: fgrep '$release' version.php

Take site out of maintenance mode:

php admn/maintenance.php --disable

Site no accessible via web.

Go site with browser.  Login as admin level.

Go to Notifications.  Check for updates.  Get plugin updates.

First time going through this is gonna take some time ... and you did take backups like suggested, right? ;)

Is above 100% to work ... well, what can I say ... is anything 100% sure today ... other than death and taxes? :|

'spirit of sharing', Ken


Average of ratings: Useful (3)
In reply to Ken Task

Re: changing to git on production stack + manually installed plugins

by Matt T -

What a fantastic answer, thank you so much

I'm only keeping on the same branch at the moment. Production site is in /var/www/ and is version 3.1.5

Git will be pulling 3.1.6. do you think I should manually being it up to 3.1.6 then sideload or am I good to do it from 3.1.5?

Don't worry the whole VM will be backed up and tested on staging environment before production smile 

Ill give it a go in staging! Thanks heaps!!!!


Average of ratings: -
In reply to Matt T

Re: changing to git on production stack + manually installed plugins

by Ken Task -
Picture of Particularly helpful Moodlers

Welcome.

The whole purpose to git is like rsync ... 'get' only what you need.   So production in /var/www/ being 3.1.5 is behind in updates for that series.   Perfect and ready for a git pull.  You might save the last git command (checkout) for executing after you get .git copied to html.

I don't use sudo ... root ... and remember you executed as root user ... cause new files will be tagged as belonging to root user.

About backups ... one is only updating moodle code ... the only other thing that is touched is the DB.   Would think restoring the VM is the type of restore one would do for a massive/big issue with the entire OS - remember, that would also restore PHP and MySQL/MariaDB and whatever config files you tweaked last.  Me, I hate loosing ground gained.

I've little bash shell scripts (one for update and one for upgrade), both of them run site cron first (hoping Moodle cleans up some junk first), then they do a backup of code and sqldump of DB - date/time stamped ... then enter the git stage for updating or upgrading.  I also rsync data directory on a daily basis.

If something goes wrong, I've only the pieces I need to restore Moodle right on the server.

Feel free to mark posting useful! smile

'spirit of sharing', Ken


Average of ratings: Useful (1)
In reply to Ken Task

Re: changing to git on production stack + manually installed plugins

by Matt T -

Thanks again.

About backups ... one is only updating moodle code ... the only other thing that is touched is the DB.   Would think restoring the VM is the type of restore one would do for a massive/big issue with the entire OS - remember, that would also restore PHP and MySQL/MariaDB and whatever config files you tweaked last.  Me, I hate loosing ground gained.

You're right - it's one massive backup of the whole system. Although I do it in maintenance windows as a last resort as with our SAN it only takes about 30 secs to snapshot/restore... The site goes into maintenance mode, VM snapshotted, but then I do the backups of code, moodledata, sql etc. If for some reason I stuff up the restore the "right" way e.g. restoring from sql dumps and old code etc. there I know I've at least got a last resort to revert everything back.

Luckily 3.1 fixed everything my custom code hacked so nothing left smile 

Just to clarify - there's no manual intervention I need regarding the extension folders I manually add in? Git pull won't look at them and demand I push them to the server as changes or something? 

Average of ratings: -
In reply to Matt T

Re: changing to git on production stack + manually installed plugins

by Ken Task -
Picture of Particularly helpful Moodlers

Moodle has what it calls 'mdeploy' ... which is from the moodle code perspective ... an update to all plugins.  After your git update/upgrade, login to site, go to Notifications, Check for Updates.

Your site should find additional plugins that are in need of updating if maintainers of plugins have kept their work up to date.   Many times that's only  a version.php file for the plugin.

Yes, even that could be automated, IF one knew there the author kept their code in the git system.   Cept it would mean kinda doing the same thing you did for core, only in the plugin directory.

Not that it's fair to compare, but Joomla in the newer versions of Joomla maintains a table for such URL's ... when you update core, one also has a button in admin to check the plugins for updates and those are updated separately.   WordPress, has a wp-cli script which is worth looking into ... it has several commands that updates not only core code but checks the plugins one has for the WP site - checks compat for version - and then acquires those as well.

So yes, there is room for improvement in Moodle.

Advise installing moosh on your site.   Automate it's ability to search Moodle plugins for versions available and run that script just prior to upgrading the Moodle core code.  IF you find a plugin that hasn't kept pace, it might be the culprit in an upgrade failing.

If everything were perfect in Moodle, there would be no need for you!  Hey, that's why you make the big bucks! (yeah, right!) ;)

'spirit of sharing', Ken


Average of ratings: Useful (2)
In reply to Ken Task

Re: changing to git on production stack + manually installed plugins

by Matt T -

Hi Ken, just an update.

I followed your instructions and it worked perfectly. The production site is now running 3.1.6 via Git. The sideload method you suggested worked great!

got a small little bash script to automate the whole shabang... maintenance mode on, dump the database, back up the code, git pull, cli/upgrade.php and maintenance off.

 Works beautifully. We'll never fall behind on security fixes again

Thanks (once) again

Matt

Average of ratings: -
In reply to Matt T

Re: changing to git on production stack + manually installed plugins

by Ken Task -
Picture of Particularly helpful Moodlers

Good to hear!

One of these days, Moodle might actually package .git .gitignore with the zip/tar.gz's for download. ;)

Just one more reminder ... install moosh ... check the plugins prior to pulling the trigger on your script.   A plugin not compat could cause some headaches. :\   And, if you run Essentials theme, read up on the docs for that as it has been the culprit in updates/upgrades (infinite loop for admins).

'spirit of sharing', Ken


Average of ratings: Useful (2)
In reply to Ken Task

Re: changing to git on production stack + manually installed plugins

by Matt T -

Moosh installed and will keep in mind.

We do run Essentials. I did have an issue with the latest release (initial upgrade was from 2.9 branch > 3.1.5, git'ified upgrade was from 3.1.5>3.1.6).

After that first upgrade, the top navbar stopped working - when users rolled over it, the dropdown menu opened. when they went to select an option on the dropdown menu, it disappeared. I worked out it was caused by a small 'gap' between the menu and the navbar-  when users moved mouse over the 'gap' it closed it. manually hacked CSS to fix it - set to 0em. Couldn't find anyone else with this issue. Other than that no issues. Will keep it mind and make sure I extensively test in staging env.

One "last" (probably not)... question. Is the process as problematic when you are 'upgrading' to later builds (but same version). Say upgrading from 3.1.6 20170502 (made up) to 3.1.6 20170511 (also made up). Do you still need to run upgrade.php to upgrade the DB or is it just changing the source code moodle runs off for bugfixes etc? Is plugin compat still a big issue? Expecting a 'yes' but may as well ask.

Suppose it comes back to test...test...test... Let's not get started on php7 plans... luckily ubuntu 14.04 still has a few years of php5 support for us to worry about that smile It's what's holding us back on a 3.2/3.3X branch upgrade actually - ubuntu official repos only have php 5.5.9 for trusty (14.04) and php7 for xenial (16.04). I'm not sure if using other third party repos is a great idea vis-a-vis support and stability. We have too many plugins to worry about potential php7 issues at this point.

Matt

Average of ratings: Useful (1)
In reply to Matt T

Re: changing to git on production stack + manually installed plugins

by Ken Task -
Picture of Particularly helpful Moodlers

Moodle 3.1 is long term support (May 2019)  but has reached a stage where it's no longer getting code fixes ... security only.   Fixes will end May 2017.

https://docs.moodle.org/dev/Releases

Now that you are under git, you won't be setting branch nor check out until you are ready to upgrade.   You will be updating only.   The 3.1.6 you have right now is the highest at present, but if there is any issue security wise, it will be getting those fixes and the version will show 3.1.6+

Normally, the very last update one performs, just before upgrading (which is the advised first step) will take only a matter of minutes and very little change to code ... version.php flie is one that is changed on every git pull.   There are other version.php files that will also be touched.

The upgrade.php script updates the database ... thus important to run it even for a minor update.

As far as strategy for upgrading ... now that you are under git, you can 'march a moodle'.  In your case you could upgrade to 3.2 stop ... check things out briefly ... and prep for major changes to DB ... before git upgrading to 3.3.x.

Before, 'marching a moodle' via ftp was/*is* painful and very prone to human error ... even if scripted.   Now your code directory stays and changes/deletions/additions happen via git - basically pulling in.  'Marching' ... while you won't see anything official that states that's a good idea, one only has to pay attention right now to forums as folks have attempted 'hyperjumps' ... skipping versions.   Some manage to pull it off, many do not - using old methods.

As is always a good practice, a backup of DB (sql dump) and tar ball of code be taken just prior to any update or upgrade.   I have those commands at the top of my little update and upgrade bash scripts with version number and date/time stamp so as not to over write the little or big updates/upgrades.

Plugins are something to check when upgrading ... normally, in an update one is staying in the same series and the only thing plugins might actually acquire is a new version.php file.

As far as OS and PHP 7 ... PHP 7 is worth the effort + 16.04 is now Linux Standards Base for some things - which I think is a plus.

'spirit of sharing', Ken

Average of ratings: Useful (2)