Best way to use Git

Best way to use Git

yazan Kieran Briggs -
Yanıt sayısı: 12

Hi.

I'm the sole developer on our Moodle VLE and currently use git to keep my local development and production server in sync.  I forked the moodle repo and then created a branch "MyPD" and in that branch, i've added a number of plugins, my theme etc.  I've not changed any core code.  I then push this back up to the remote and when I update it I just pull the lastest version of my branch down to the production server.

This has worked fine while I'm not trying to upgrade the version of Moodle, but can someone tell me how I could go about upgrading the version of Moodle in this situation?  The version is currently 3.5.2 but I want to move to 3.7 . If I just check out the 3.7 branch I'll lose all the plugins that i've installed.

Thanks for any help.

Kieran

Kieran Briggs yanıt olarak

Re: Best way to use Git

yazan Davo Smith -
Core developers 'ın resmi Particularly helpful Moodlers 'ın resmi Peer reviewers 'ın resmi Plugin developers 'ın resmi

Hi Kieran,

We have a similar situation with a lot of different customers whose code we manage.

There are lots of different ways to handle this situation (so I'm sure others will weigh-in with alternatives), but the way we handle it is as follows:

  • For minor releases (3.5.2 => 3.5.3, etc.):
    •  we checkout the latest Moodle code (MOODLE_35_STABLE)
    • then checkout our branch
    • then create a new branch based on our current branch (all our branches are named m_3.5.2_customer, in order to keep track of our exact version)
    • then use git rebase to move our new branch on top of the latest MOODLE_35_STABLE
    • deploy this new branch
    • Note, an alternative is to simply rebase the current branch on top of the latest MOODLE_35_STABLE, but after doing a rebase you have to do a push --force, which can cause issues if other people were working on that branch
  • For major releases (3.5 => 3.7):
    • Create a new branch based off the Moodle code (MOODLE_37_STABLE)
    • Download the latest versions of all plugins and add them to the code (many of the plugins will have newer versions released, so it is easier to install them all fresh, rather than trying to go through and add the updated versions)
    • Commit each plugin as a separate commit (makes it a lot easier to review the list of custom plugins)
    • Cherry-pick any core changes + custom plugins (sometimes taking an opportunity to squash the commits if there are a lot of minor bug fixes we don't need to retain the history of).

Depending on how many plugins there are to install, the major upgrade prep rarely takes more than 30-60 minutes for a customer.

I hope that helps.

Kieran Briggs yanıt olarak

Re: Best way to use Git

yazan Andreas Grabs -
Core developers 'ın resmi Particularly helpful Moodlers 'ın resmi Peer reviewers 'ın resmi Plugin developers 'ın resmi Translators 'ın resmi

Hi Kieran,

as Davo already said there are a lot of ways you can hold your plugins and moodle uptodate.

One way could be that:

  1. Create a repository where you put all your plugins and store it as a branch suitable to your moodle version.
  2. Create a repository with your moodle and as you already have done create a branch of your own suitable to your moodle version.
  3. Merge the repository from 1) into the repository from 2)

As commandline it should look like this.

# creating the plugin repository
git init
# Create a branch for your moodle version e.g. 3.7
git checkout -b mdl_37
git remote add origin <your-git-url>/myplugins.git
# Put your first plugin into this repository. Do this in the correct directory, as it should be later.
git add .
git commit -m "added plugin xyz"
# Do the same for all plugins
# push the repository to the git server
git push origin mdl_37

# clone moodle in another repository
git clone -b MOODLE_37_STABLE https://github.com/moodle/moodle.git moodle
# create your own branch
git checkout -b mdl_37
# Add the first repository as remote and fetch it
git remote add myplugins <your-git-url>/myplugins.git
git fetch myplugins
# merge the plugins
git merge --allow-unrelated-histories myplugins/mdl_37
# Push it to the git server
git push origin mdl_37

This way, you can update your plugins and your Moodle version separately and always see which plugins you use.

Maybe that is a way for you too.

Best regards
Andreas

Kieran Briggs yanıt olarak

Re: Best way to use Git

yazan Mark Sharp -
Core developers 'ın resmi Particularly helpful Moodlers 'ın resmi Plugin developers 'ın resmi

We keep the core moodle repo and our changes separate and merge them together using rsync. It helps make sure we don't accidentally make changes to core moodle.

So our structure & process is something like:

~/moodle (moodle repo, or download)
~/pluginsthemesetc (same directory structure as Moodle)
$ rsync -avz ~/moodle/ /var/www/html/
$ rsync -avz ~/pluginsthemesetc/ /var/www/html/
$ php /var/www/html/upgrade.php

I use bash scripts to manage the process.

If I absolutely have to change anything in core, then I include and run patch files, but I keep these to an absolute minimum. I have one patch file for a community plugin atm, and that's it.

Kieran Briggs yanıt olarak

Re: Best way to use Git

yazan David Mudrák -
Core developers 'ın resmi Documentation writers 'ın resmi Moodle HQ 'ın resmi Particularly helpful Moodlers 'ın resmi Peer reviewers 'ın resmi Plugin developers 'ın resmi Plugins guardians 'ın resmi Testers 'ın resmi Translators 'ın resmi

You might be able to "replant" your MyPD branch onto 3.7 with the git rebase --onto command.

Basically something like

# git rebase --onto MOODLE_37_STABLE MOODLE_35_STABLE MyPD

which does something like "take all commits on the MyPD branch that are not present on the MOODLE_35_STABLE branch and re-apply them on top of the MOODLE_37_STABLE branch". Read the fine manual for more details.

Like others already mentioned above, there are many creative ways you can use Git to compose code for your customized Moodle sites. In HQ we ended up with maintaining a separate branch for every single plugin or core modification. We follow a strict naming scheme for these branches which allowed us to have helper scripts for common tasks such as "take that one particular core hack we used on moodle.org and re-use it on another site" etc.

Kieran Briggs yanıt olarak

Re: Best way to use Git

yazan Colin Perepelken -
Aside from the what the others have mentioned, if you aren't modifying core code, I find using submodules to manage plugins works very well. https://git-scm.com/book/en/v2/Git-Tools-Submodules

To add a submodule to a repository, use:

git submodule add [source_url] mod/myplugin

You can update all submodules at once using:

git submodule update

It is very easy to keep the plugins and core Moodle code separate this way, and could make upgrading simpler in the future.

Colin Perepelken yanıt olarak

Re: Best way to use Git

yazan Kieran Briggs -
I was thinking about Submodules. If I clone a branch with submodules will it bring the modules with it too. So for example, as I said, I have a localhost development machine and then the production server is where I just use a git pull to update. If I update the submodules on the remote will it pull them down on a git pull onto the production server?
Kieran Briggs yanıt olarak

Re: Best way to use Git

yazan Colin Perepelken -
Make sure you add your submodules to the main repo (at the Moodle root). Then when you want to deploy to production for the first time, you would:
1. Git clone / git pull your master (moodle root) repository.
2. Run git submodule update --init

To answer your question, yes for any subsequent changes you can just push / pull the master repo and the submodule changes will be pulled down as well. However you will have to make sure these submodule changes have been committed to the master repo as well as to the submodule, so Git knows which submodule commit to pull. Submodules will be checked out to a specific commit, rather than a branch.
Kieran Briggs yanıt olarak

Re: Best way to use Git

yazan Andreas Grabs -
Core developers 'ın resmi Particularly helpful Moodlers 'ın resmi Peer reviewers 'ın resmi Plugin developers 'ın resmi Translators 'ın resmi

Hi Kieran,

there are a few things you keep in mind when using submodules.

  1. You never know whether or not the submodule is available while updating your git on the production mashine
  2. You never know whether or not the git origin of your submodule is consistent.

So if you really want to use submodules in a production environment, you'll need to create your own repositories for all your plugins and then use those repositories as submodule in your main repository. Otherwise you might end up in a broken productive system.

Best regards
Andreas

Andreas Grabs yanıt olarak

Re: Best way to use Git

yazan Brian Merritt -
Particularly helpful Moodlers 'ın resmi
Good points all.

We use submodules and generally clone repositories only where we have to make changes (which we try not to do) or where the plugin is not receiving love and care.

Each new Moodle major release just requires adding the submodules (which can be done in php by reading the .gitmodules from the previous release and running a git submodule add script to pull them in)

We use php deployer https://deployer.org/ for deployment, with branches MASTER (production quality test), STABLE (production quality deploy to live) and FEATURE (development quality).

Having a staging server is essential as you can run a test deploy and make sure it works before going live.

Because plugins can be nested, we tend to use often git submodule update --init --recursive

A few other things to keep in mind re submodules

1. when git adds the submodule the branch may be wrong. make sure you checkout the right branch and commit that
2. if changing the branch (maybe the plugin has a MOODLE_37_MASTER branch) so check releases whether they have a more appropriate branch available
3. the git commit of the main Moodle sets the submodule to a specific commit. If the remote plugin is updated you need to do a git pull of the submodule and commit that change back to Moodle main.
Brian Merritt yanıt olarak

Re: Best way to use Git

yazan Brian Merritt -
Particularly helpful Moodlers 'ın resmi
Oh yeah - each Moodle version we keep moving forward, so Moodle master get's branched off when 3.8 goes into development, so we have Moodle 3.7, 3.8 and master moving forward over time (which then becomes 3.8, 3.9, and master). It takes a bit of time, but makes security fixes and future development testing much simpler.
Kieran Briggs yanıt olarak

Re: Best way to use Git

yazan Pawan Pandey -
Hi there!

I think in your architecture you have to utilize Git Submodules and Subtree. Submodule can be used for external plugins in which you are performing you changes,
whereas Subtree can be utilized for managing dependencies which doesnt include changes by you.

For that you have to find source control url for all external plugins, clone all those plugins in separate plugins, using tools like Sourcetree which helps you to make the state of plugins as same in your Moodle 3.5.2 , use code comparison tools like Winmerge to find plugins which are changed by you. Create submodules for those with changes, and subtree for those without changes.

For moving from One Major Upgrade to Another, what i can see their is you can write some scripts to intialize the submodules and subtree in the new system. For core changes, i will recommend try your best to keep them at minimum. Yeah you can cheery pick to move those core changes, but i am not sure whether that will be robust approach. Better to keep one document, to maintain core changes and perform on new version.


Only problem with above architecuture is that it requires lot of Git skills, and in case of multiple people working together

Moving core changes wont be easy. The best one is stick to Moodle LTS version currenlty 3.5 , then wait till May 2020 for latest Moodle 3.9 version so that you dont have to do such much operations frequently.