General developer forum

HEADS UP: Backporting policy for Moodle core

 
Dan at desk in Moodle HQ, Perth
HEADS UP: Backporting policy for Moodle core
Group Core developersGroup Moodle Course Creator Certificate holdersGroup Moodle HQGroup Particularly helpful MoodlersGroup Plugin developersGroup Testers

Hi,

In the integration team we have felt that the rules for backporting changes to stable branches were a little unclear and lacking consistency. Additionally we have received feedback that the stable branches are not 'stable enough'.

To address this situation we've updated the Integration Review docs page to include a new backporting policy and the process for requesting an issue to be backported.

We intend to start applying this policy immediately, but we welcome comments and any suggested revisions to this!

thanks,

Dan and the rest of the integration team

 
Average of ratings: Useful (2)
Picture of Kris Stokking
Re: HEADS UP: Backporting policy for Moodle core
Group Plugin developers

Dan, thanks for the clarification on backporting and for your continual improvement of stable branches. As the release manager for Joule, I know all too well the delicate balance between functionality and stability, not to mention the time commitment on "older" branches (fortunately, we have very few of those). It is a difficult conversation to make end-users wait for the right time to receive updates. Unfortunately for hosts, when that conversation refers to a 6 month timeline to fix a "bug" that has been listed as an improvement (the antithesis of your polite note), the conversation becomes immensely more challenging. Moodlerooms and many other partners/institutions cannot simply update to the latest-and-greatest release immediately after its launch as there are months worth of work to upgrade internally developed code and 3rd party dependencies. Nor can every institution completely understand the impact of a change 2 months after it lands in master. I fear (maybe unjustly) that hosts may be put in an awkward position when a fix is not backported, and I'd like to make sure we make all attempts to pass the option down the line.

Your Stable releases are similar to our Maintenance Packs, and we have similar policies for what can and cannot be included. Our goal is to have zero unwanted impact to end-users as it will almost definitely cause extra work - increased support inquiries, documenation updates, etc. But we counter that appropriately with risk. We do everything to ensure that clients have the option to move at their pace. Ultimately, this ensures backward compatibility while still giving individual administrators the ability to decide. If they want to wait until an "improvement" has come out of beta, they can wait. If the improvement is mission critical and they cannot wait, they have the option to expedite. And at the end of the day we have informed users who understand the implications of their decision, and are knowledgeable about the situtation if they run into problems.

We do this almost exclusively through the use of $CFG options because we can control the configuration files. It may not be the best option for fixes from Core, but I think if we got into a habit of communicating experimental backports through release notes it would be comfortable to site admins. And there are always Global Settings, particularly Experimental Settings which by definition scream use-at-your-own-risk. It's not a silver bullet in all cases, but it does gives us better control over rolling out impactful changes, and our clients are extremely happy as a result. Because at the end of the day, no one wants to be in the unfortunate position of telling their end-users (or paying customers) that a critical backport is unavailable because they missed the review window, or because it was reviewed and ultimately denied (in a smoke filled room no doubt smile)

Here's what I'd like to see (the TL;DR version):

1. Think creatively how impactful changes can still be backported using toggles/settings to control backwards compatibility.
2. Remove the 2 month "window" for backporting. Requests should be considered based on need, feasibility and effort.
3. If a backport is reviewed and denied, please give an appropriate reason behind the decision (e.g. potential data loss, unavoidable backward compatibility break, etc.). I assume you would do so, but it wasn't listed in the policy.
4. Mitigate regression testing with increased automated functional testing!

 
Average of ratings: -
Dan at desk in Moodle HQ, Perth
Re: HEADS UP: Backporting policy for Moodle core
Group Core developersGroup Moodle Course Creator Certificate holdersGroup Moodle HQGroup Particularly helpful MoodlersGroup Plugin developersGroup Testers

Hi Kris,

Thanks for your input.

Think creatively how impactful changes can still be backported using toggles/settings to control backwards compatibility.

I agree, its a good point. But I think that we have to be mindful of problems with this too - some of the biggest complexity for Moodle users and developers can be the amount of settings we have. I'd love to know how many combinations of settings we have, it'd be mind blowing and variations in these settings is a common source of bugs.

Remove the 2 month "window" for backporting. Requests should be considered based on need, feasibility and effort.

Well, can you alert us to examples of where this policy is failing and then we can review it. I hope that you can believe that we will try to be pragmatic about this when there is an obvious need. Setting these boundaries can be quite helpful to keep us focused

If a backport is reviewed and denied, please give an appropriate reason behind the decision (e.g. potential data loss, unavoidable backward compatibility break, etc.). I assume you would do so, but it wasn't listed in the policy.

Agreed, I edited the policy to add this and we've already been commenting on backport requests like this.

Mitigate regression testing with increased automated functional testing!

200%, but it'll take us some time to get there. We improve each day.

 
Average of ratings: -
Tim at Lone Pine Koala Sanctuary
Re: HEADS UP: Backporting policy for Moodle core
Group Core developersGroup Documentation writersGroup Particularly helpful MoodlersGroup Plugin developers

Hmm. I wonder if that is a way to quantify whether a particular change (whether it is labelled a bug fix or improvelment) is 'safe' to back-port.

If the change does not break any of the existing Behat tests of the stable branch, then it is safe to back-port, since it does not change the way any key functionality works.

(I think that does not acutally work, even when we have lots of Behat tests, but it is worth thinking about.)

 
Average of ratings: -
Dan at desk in Moodle HQ, Perth
Re: HEADS UP: Backporting policy for Moodle core
Group Core developersGroup Moodle Course Creator Certificate holdersGroup Moodle HQGroup Particularly helpful MoodlersGroup Plugin developersGroup Testers

I'm agreed its not an especially good metric Tim. (Although there are some advantages - many improvements and new features need different documentation, where as bugfixes generally don't change documentation).

Taking off my Moodle HQ/Integrator hat for a second, my personal bias is to have far less changes of any type going into the stable branches.

I think that the combined effect of all the bugfixes between point releases can make a point release upgrade sufficiently scary that some institutions and moodle partners don't do it lightly (if at all). In fact when I was managing a large Moodle deployment I backported security fixes myself rather than risk a point release upgrade and i've heard of other institutions doing that too.

It makes me sad that this is the case when I see the amount of resources we put into maintaing the stable branches at HQ. It would be interesting to me to see us doing less back-porting and more work moving forward, adding regression tests and bolder improvements with our new tools evolving. It could even be an opportunity for value-add by partners they could backport things which are important to their clients as a piece of differentiation.

But, hopefully i've just got a jaded opinion of the communities hesitance to upgrade on stable branches, and at HQ we're catering best to the needs of the community and Moodle Partners who, after all - pay for me to keep working on this stuff. smile

 
Average of ratings: -