This reminded me a question I wanted to raise for some time: How to install and maintain contributed extensions as submodules when they don't have the same branch (yet)?
Here is the question in detail:
- First I create a local Moodle repository to track the remote branch origin/MOODLE_38_STABLE - as in https://docs.moodle.org/38/en/Git_for_Administrators#Obtaining_the_code_from_Git.
- Then start installing submodules as explained in
https://docs.moodle.org/38/en/Git_for_Administrators#Installing_and_maintaining_contributed_extensions_using_Git_submodules. The problem is, not all submodules respond positively to 'git checkout -b MOODLE_38_STABLE origin/MOODLE_38_STABLE', obviously the branch is not there.
How do I still take them in to my repository? The idea is that the repo as itself could be the code directory of Moodle. At any time after that I can march the repo forward as the origin is updated with 'git submodule foreach git pull'.
Is this possible? Or do I have to fall back to origin/MOODLE_37_STABLE in those submodules? How do I know, when the origin get the branch MOODLE_38_STABLE on a later date?
P.S. If this question is not related to the OP https://moodle.org/mod/forum/discuss.php?d=396152 please do split.
Anyway, we are not quite at the point of managing our entire installation with Git, so some of this stuff is a bit beyond my knowledge. But generally, my understanding is that a plugin that lives in someone else's repository can have any arbitrary branching convention (or none at all), and so there is never a guarantee that one method will work for any other plugin.
In your situation, are you considering what to do in the following kind of situation? Say:
- you have Moodle 3.7 installed,
- with a 3.7 version of a plugin; and
- you are upgrading to Moodle 3.8,
- and you want to get the 3.8 version of the plugin, but there is no 3.8 branch for that plugin yet
Thanks for asking for clarifications. This is still an idea, don't know whether it holds water. May be this dialog will help to refine the idea. This time let's begin with MOODLE_37_STABLE.
- The main idea is to create a master repo of my own (well, distributed repos are at the core of Git)
-- the base of which is https://docs.moodle.org/37/en/Git_for_Administrators#Obtaining_the_code_from_Git
-- a number of https://moodle.org/plugins/ will be added to it following https://docs.moodle.org/37/en/Git_for_Administrators#Installing_a_new_extension_into_an_existing_Moodle
Yes, I am talking of the official Moodle repo on git.moodle.org and the plug-in repos on GitHub. No own plug-ins nor code changes (for the time being).
- Then I need only to clone that master repo 'git clone /path/to/master /apache/docroot/clone' to start creating a new Moodle instance
- When it comes to maintenance
-- to update as minor releases come, 'git pull' for the core, https://docs.moodle.org/37/en/Git_for_Administrators#Maintaining_Git_submodules which recursively update the submodules.
This will work, provided that all the submodules have a MOODLE_37_STABLE branch.
-- to upgrade (major release, say MOODLE_38_STABLE) execute two sets of commands similar to the above.
Now that will lead to errors in those submodules which still don't have a MOODLE_38_STABLE branch. The question is how to handle that situation? I mean, I want to go to the next release but some somemodules don't have it. How to proceed?
I have read the Git for Administrators page a few times in the past, and that is why I remember the thing about branch name conventions.
my understanding is that a plugin that lives in someone else's repository can have any arbitrary branching convention (or none at all)
That's correct and common.
The problem is, not all submodules respond positively to 'git checkout -b MOODLE_38_STABLE origin/MOODLE_38_STABLE', obviously the branch is not there.
I do not use submodules personally so I am just guessing here. I am wondering, do all submodules need to share the same branch name as their parent module? I thought they were pretty independent. So that you could have the parent (moodle core) checked out on the MOODLE_38_STABLE branch, and yet each of the plugins / git submodules could be checked out on their own relevant branch. No?
If that is the case, I need an alternative to these commands:
$ cd path/to/extension
$ git checkout -b MOODLE_37_STABLE origin/MOODLE_37_STABLE
Branch MOODLE_37_STABLE set up to track remote branch MOODLE_37_STABLE from origin.
Switched to a new branch 'MOODLE_37_STABLE'
If that is correct, the part of the Moodle Doc I have quoted in my earlier post is misleading.
Would appreciate confirmation from those who have the insight.
For my institution I maintain a fork of the main Moodle project. My fork is different specifically because I have added submodules. When I pull new code from Moodle it does nothing to the submodules. It only updates the core.
To upgrade the submodules I look at what the latest release is for the version of Moodle I have. Then I go into the submodule and check out that release. Then I go back to my main project and commit the change. This allows me to set up an entire installation locally, then push my forked Moodle (with submodules) to our git server so that I can pull the new code to our installations. When I do that I will have to pull my fork and then run `git submodule update`. That command will checkout the desired commit for each of my submodules, but it can only do that because I already set it up on my fork.
The point of using the submodules is not to upgrade third-party code based on your Moodle version. It's really about specifying, in your main project, which commit you want each of your submodules to be on. Is that helpful?
Many thanks for the detailed description of your workflow. That confirms my new plan (in my previous post).
The old plan, in the earlier posts, will not work. Which means the documentation https://docs.moodle.org/37/en/Git_for_Administrators#Installing_and_maintaining_contributed_extensions_using_Git_submodules, also quoted in https://moodle.org/mod/forum/discuss.php?d=396476#p1598412, needs an overhaul. I won't touch it, don't claim to understand enough.
Let me put the question differently. You wrote:
> A main repository and its submodules are separate in the sense that upgrading the main project will not automatically upgrade the submodules, ..
But if I enter the following command in my main repo (Moodle)
$ git submodule foreach git <command> [<args>]
won't git walk through all the submodules and execute 'git <command> [<args>]' in them?
If that is the case, what I need is to find out '<command> [<args>]' which will update (pull?) every submodule to latest version which is still compatible with the original Moodle release!
I know, not all plug-in developers follow the same pattern of branching or tagging. I am looking for a system of branching and/or tagging the plug-ins so that the identical git command will update the whole super repo. Can you (or anybody else) think of such a system?
2 cents to add to confusion? :|
git pull does pull core code ... whatever stable branch tracking.
IF a plugin was installed with git, a cd /to/plugin/ git pull acts the same way, only for the stable branch of the plugin - automagically, I thought.
Thus no special git command needs to be used ... cept a master for site + plugins .. and that could be combined with DB backup and code archive just prior to pulling the trigger and after if that's desired.
echo '3.4.9 (Build: 20190513)'
# tar -cvf /home/backup/m34/moodle-code-349-$(date +%Y%m%d%-H%M%S).tar ../moodle34;
# tar -cvf /home/backup/m34/moodle-data-min-349-$(date +%Y%m%d%-H%M%S).tar /home/moodle34data/filedir;
# mysqldump -u root -p'' moodle34ssl > /home/backup/m34/moodle349ssl-db-$(date +%Y%m%d%-H%M%S).sql;
php admin/cli/maintenance.php --enable;
php admin/cli/upgrade.php --non-interactive;
php admin/cli/maintenance.php --disable;
Now upgrades, different story but only in git commands to point to upgrade tracking. And that's where plugins installed with git might differ in each plugin, me thinks!
One updates more than one upgrades so until one does upgrades to mods/blocks/addons via git manually a time or two to assure the repo for the plugin behaves as expected, then an upgrade to site ... auto git pull for code, manual cd /mod/ git branch -a check, then if needed change tracking, etc.
I also had a similar script to set up the code base, so it would clone Moodle and set the appropriate branch, then would do the same git clone for each plugin, and where appropriate would cd into the plugin folder and set the relevant branch. Not much use for a single site as I had to maintain the script with all the plugins and relevant branches (where it didn't just use master), but if you want to spin off quick identical test/WiP dev sites, it was quite handy.