using Git with Moodle

using Git with Moodle

by Hittesh Ahuja -
Number of replies: 7

This is how I envision our structure to be ;


Moodle base ( 2.6.3) + some moodle core changes + extra blocks,plugins,mods,etc 


This has been saved in git as its own branch ( something like Moodle_26_MODIFIED) under our repo. 


Now , come summer, if we need to upgrade to say Moodle 2.8.2 whats the best way to merge the 2.8 code with my custom branch so that it replaces 2.6 files with 2.8 files but leave alone the custom plugins and the core changes.


I've thought of using git merge where if i am currently on the Moodle_26_MODIFIED branch i do something like :

git merge --strategy-option theirs MOODLE_28_STABLE

but this ends up in giving me a lot of merge conflicts.

The other way I was thinking of doing was to create a seperate branch that holds all of my core changes + extra blocks + plugins and then merge that into the latest moodle source that i wish to deploy. 

Does anyone else have any ideas on how to better  merge changes for a moodle installation that has a fair bit of core changes.


Any help appreciated


Hittesh  

Average of ratings: -
In reply to Hittesh Ahuja

Re: using Git with Moodle

by Davo Smith -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

http://tjhunt.blogspot.co.uk/2014/01/moving-ou-moodle-code-to-moodle-261.html

Or, for a cleaner git history, you can try rebasing your branch onto each new version of Moodle as it is released (but, with the downside that this breaks the branch for anyone else who has an earlier copy of it - they will need to throw away their copy of the branch and pull a fresh copy of your rebased version).


Average of ratings: -
In reply to Davo Smith

Re: using Git with Moodle

by Neill Magill -
Picture of Core developers Picture of Peer reviewers Picture of Plugin developers

You could always create a copy of your branch and rebase the copy to get around that. It would also have the advantage that if anything went horribly wrong when the rebase/fixing of any merge conflicts from it you can get easily back to a state that you can try again.

If you do not have too many commits in your modified branch you could also try cherry picking them into a new branch based on 2.8.

Average of ratings: -
In reply to Hittesh Ahuja

Re: using Git with Moodle

by David Mudrák -
Picture of Core developers Picture of Documentation writers Picture of Moodle HQ Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers Picture of Plugins guardians Picture of Testers Picture of Translators

Moodle HQ use something similar for maintaining the code for all Moodle community sites (such as this moodle.org). Our model is slightly different from what the Open University uses and seems to work well for us, too.

Basically, we have a clone of moodle.git. In a certain and well defined point (best identified by some upstream tag), we create our own tag that acts as a base point for the whole site composition. Let us say this base tag is called ourmoodle-2.8-base and it matches the commit tagged as v2.8.0.

The basic idea of our model is that every single local hack/customization and every single additional plugin goes into its own "feature" branch. All these branches are created off the base tag. We have certain rules on naming these branches, which helps to administer it all.

Let us say there is a branch ourmoodle-2.8-theme where the site theme plugin is committed (we commit just snapshots of additional plugins, they have their own development history in their native repositories) and another branch called ourmoodle-2.8-login-page-hack with some customization.

We create a branch called ourmoodle-2.8-test and we merge all feature branches ourmoodle-2.8-* as well as upstream MOODLE_28_STABLE branch to it.

This -test branch can be deployed on our notebooks and the test server and it helps us to be sure that we all share 100% same version of the code.

Once we are happy with testing, the -test branch is again merged to another ourmoodle-2.8-deploy branch that is then deployed to the production site.

When a new 2.8 stable version is available, we simply merge MOODLE_28_STABLE into the ourmoodle-2.8-test branch (and then, after testing the upgrade etc), we merge that into ourmoodle-2.8-deploy again. This can be done easily for every weekly build, if we want to.

When doing the major version upgrade, say from 2.8 to 2.9 in this example, we principally use git rebase --onto. We are aware of every single modification done in our 2.8 site (because they all are tracked on separate branches) so we can re-create ("replant") these feature branches in the new version. In other words, we create a new base tag ourmoodle-2.9-base and we create a new branch ourmoodle-2.9-login-page-hack from it. We know that all the commits in the range ourmoodle-2.8-base..ourmoodle-2.8-login-page-hack need to be re-applied to this new branch. We usually use git-rebase, git-cherry-pick or git-format-patch && git-am to do that. If there is a conflict, we know that upstream has changed and we need to investigate further. Even if the patches apply without the conflict, testing is still needed as there might be some change elsewhere that does not allow the customization work any more.

Average of ratings: Useful (3)
In reply to David Mudrák

Re: using Git with Moodle

by Andrew Lyons -
Picture of Core developers Picture of Moodle HQ Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers Picture of Testers

Having experimented a lot with the different ways of doing this, David's way is the best in my opinion.

I wrote a blog post on this strategy 18 months ago (http://thamblings.blogspot.com.au/2013/07/upgrading-moodle-from-git.html), and I know that the people that I used to work with still use it. The strategy was developed largely from the ideas of several core Moodle Developers, and based on the experiences of other organisations (including the Open University).

I'd highly recommend looking into this.

Things to note:

  • David mentions the base branch. It should always be based on the .0 tag (e.g. 2.8.0; 2.7.0; 2.6.0). It should never be a point release (e.g. 2.8.1; 2.7.3).
  • David discovered that using a forward-slash as a separator between the instance+version and the feature name is much more powerful (e.g. instead of ourmoodle-2.8-login-page-hack, use ourmoodle-2.8/login-page-hack). This allows you to do such magic as git push myrepo/ourmoodle-2.8/*
  • I'd also recommend that you create a task in your project's bug tracker for the next release. E.g. If you are running 2.8 now, then your next major release will be 2.9. Under that project, create a sub-task for every feature branch that you merge (e.g. Merge login-page-hack branch). When it comes to your upgrade time, you can then go through that list and tick them off, one-by-one. If you no longer need your login page hack, then you can comment on the subtask to say why and you have a record for future reference.

As I say, I would highly recommend this approach.

Andrew

Average of ratings: Useful (1)
In reply to Andrew Lyons

Re: using Git with Moodle

by Hittesh Ahuja -

There has been some really helpful tips here !! 

thanks to each and everyone for their input.

 I like some bits of Tim Hunt but mostly would follow David's structure. I shall definitely reply here or blog about it once I'm ready with my structure for posterity  smile


Thanks again. 

Average of ratings: -
In reply to Andrew Lyons

Re: using Git with Moodle

by Hittesh Ahuja -

There has been some really helpful tips here !! 

Andrew, you say : 

  • David mentions the base branch. It should always be based on the .0 tag (e.g. 2.8.0; 2.7.0; 2.6.0). It should never be a point release (e.g. 2.8.1; 2.7.3).

Why is that ??? 

 I like some bits of Tim Hunt but mostly would follow David's structure. I shall definitely reply here or blog about it once I'm ready with my structure for posterity  smile


thanks to each and everyone for their input.

Average of ratings: -
In reply to Hittesh Ahuja

Re: using Git with Moodle

by Andrew Lyons -
Picture of Core developers Picture of Moodle HQ Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers Picture of Testers

Hi Hittesh,

The reason that I would really recommend always basing on the .0 release is because it makes your life much easier in the long run. In my experience, most of the feature branches you're creating are for plugins which you want installed. These do not need to be based on a point release. The other frequent type are the inclusion of hot-fixes before they are included in a point release. The less-frequent type are for the inclusion of hacks to Moodle core.

In all of these cases, there is no real benefit to basing on a point release, and doing so can complicate things at upgrade time.

By basing on the major release tag, when it comes to the next release; you can simply do:

# Fetch from our upstreams
git fetch ourprivaterepo
git fetch origin

# Checkout the feature branch for the current version of Moodle
git checkout ourmoodle-2.8/some-feature-branch

# Create and checkout a branch for our intended version
git checkout -b ourmoodle-2.9/some-feature-branch

# Rebase on that release
git rebase v2.9.0

# Push it back to our repository
git push ourprivaterpo ourmoodle-2.9/some-feature-branch

As you can imagine, the above can easily be scripted to take three things and automate this.

  • the source tag (v2.8.0);
  • the target tag (v2.9.0); and
  • a list of branches to try and rebase.

The alternative would be to use rebase --onto as David discusses. Both ways are fine, but with major tags it is generally much easier for 99% of situations, and is also much faster to do. The way that rebase --onto essentially works is to rollback each commit from the point release to the shared point (e.g. v2.8.0), and then replay all of the commits between v2.8.0 and v2.9.0.

There will be some situations where you have to base your branch on top of a point release (e.g. because your change conflicts with a change in a point release), but for the majority of situations this way is much less confusing and is generally easier and faster to update with.

Generally the occasions where you're basing on a point release are more likely to require manual intervention because of conflicts.

Andrew

Average of ratings: Useful (1)