Development/branching flow for Moodle sites

Development/branching flow for Moodle sites

by Michael Ko -
Number of replies: 6
Good day.


I am a developer on a Moodle site where we have approximately 30-40 internally-developed custom plugins (activity modules, filters, local plugins, admin tools, question types, auth, enrol, themes, blocks, etc.) and I am often working on several projects simultaneously, spanning across multiple plugins.

The problem with using a normal git branches for each feature is that different branches might contain different plugin versions, and/or DB tables, in which case, moving between branches can be problematic. 

We've also decided against a seperate git repository for each plugin, due to the deployment complexity and dependencies between some of them.

However, keeping the entire Moodle site in a repository makes it absolutely massive, and we generally rebuild the repository after each Moodle version update.

I am wondering if anyone else has these types of issues, and how to work around them? I find myself frequently building new local copies of the site. Might it be necessary to set up a separate docker container (with PHP and the DB) for each feature set?

Thanks


Average of ratings: -
In reply to Michael Ko

Re: Development/branching flow for Moodle sites

by Richard Oelmann -
Picture of Core developers Picture of Plugin developers Picture of Testers

I also have around 30 in-house plugins, from local scheduled tasks, to blocks and customised versions of 3rd party plugins

I definitely use git for each one separately and rarely have any conflicts from making changes in several different places on the same plugin without pulling first - I try hard to have a workflow that means that I push changes regularly and before I start work on anything, to make sure I've pulled it and updated it. Frequent updates to git, and frequent pulls to WIP development instances are a couple of commands and I think I would find it much easier to keep doing that than to have to rebuild a massive repository every time Moodle updates. I have a bash script to help with this and can set up a new moodle instance with all plugins simply by running that bash script, with a second one that goes through core and every plugin and pulls updates to them all.

I have the (dis)advantage of being the only developer at the Uni though, so I don't have to worry about anyone else changing something at the same time - but isn't that the idea of branches, merges, and sometimes having to resolve merge conflicts manually when they occur. Git is designed for multiple users and multiple WIP streams.

I can't imagine the complexities around trying to maintain a full Moodle branch incorporating all the plugins I use and develop as a single repository, let alone then maintaining and coordinating between multiple branches for different projects and the difficulty of merging those branches back to a master one.

Another thought would be that it may be an opportunity to look at some of the non-tech solutions - a change control process or an agile style standup so the team knows what everyone is working on, can adjust their work accordingly if it is likely to create conflicts, and more frequent pushing/pulling to keep conflicts to a minimum?

Average of ratings: Useful (1)
In reply to Richard Oelmann

Re: Development/branching flow for Moodle sites

by Rex Lorenzo -

Are you using git submodules? We manage 63 plugins across two different instance and four environments (Dev, Test, Stage, and Prod) using submodules. It makes it easy to pull in changes from plugins if they have a git repo.

Average of ratings: -
In reply to Rex Lorenzo

Re: Development/branching flow for Moodle sites

by Dan Marsden -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers Picture of Plugins guardians Picture of Testers Picture of Translators
The other thing to consider if you don't like submodules is to use git subtree instead - subtrees work quite well for us when we have multiple branches with completely different plugins.
Average of ratings: -
In reply to Dan Marsden

Re: Development/branching flow for Moodle sites

by Andreas Grabs -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers Picture of Translators

Yea, subtree is my preferred choice too.

I have to manage a lot of instances with a lot of different plugins and different versions.
One of the important things to me is, you can give your customer access to the so custumized moodle repository without access to all the plugins. That's something you can't achieve with submodules.

Another thing I like on subtree is you have the whole history at one place and you can work with it without changing into the plugin direktory.

Best regards
Andreas

Average of ratings: -
In reply to Rex Lorenzo

Re: Development/branching flow for Moodle sites

by Richard Oelmann -
Picture of Core developers Picture of Plugin developers Picture of Testers

Personally, no I'm not - I know there are strengths to that, but for now I still have everything in its own independent repo and just use a bash script to replicate a whole site, or update an existing one. It's then easy for me to just manage that script between sites for which plugins I need on a WIP or Dev or even localhost site.

Average of ratings: -
In reply to Michael Ko

Re: Development/branching flow for Moodle sites

by Michael Aherne -
Picture of Core developers Picture of Peer reviewers Picture of Plugin developers
We've also decided against a seperate git repository for each plugin, due to the deployment complexity and dependencies between some of them.

It might be worth reconsidering this decision, as it sounds like it could be the root of your issues. As Richard says there are ways to manage deployment where each plugin is in its own repository, and it'll probably be simpler in the long run that trying to manage 30+ plugins alongside the Moodle codebase in a single git repo.

Average of ratings: Useful (1)