Improved validation, API for maintainers and ability to release by tagging at GitHub

Improved validation, API for maintainers and ability to release by tagging at GitHub

David Mudrák -
回帖数：9
Core developers的头像 Documentation writers的头像 Moodle HQ的头像 Particularly helpful Moodlers的头像 Peer reviewers的头像 Plugin developers的头像 Plugins guardians的头像 Testers的头像 Translators的头像

Hello! I am very happy to finally announce availability of some new features in the Moodle Plugins directory. These improvements are mostly relevant for the maintainers who release new versions of their plugins regularly.

Improved version number validation

  • Plugins must have well formatted version number defined as $plugin->version in their version.php file. Invalid values used to raise warning only before.
  • If specified, the $plugin->requires must be Moodle core version number in the form of YYYYMMDDRR or YYYYMMDDRR.XX. Invalid values did not raise any warning before.
  • For plugins that specify $plugin->requires only in their version.php, all Moodle releases with version number equal or greater than the required one will be automatically selected as supported by default.
  • The incompatible Moodle branch specified in $plugin->incompatible is now validated and taken into account when populating the default list of supported Moodle versions (all between the required one and the incompatible one).
  • Supported Moodle branches specified in $plugin->supported are now respected when populating the list of supported Moodle versions. Only versions that pass all present conditions (supported, requires and incompatible) will be selected.
  • It is no longer possible to release a new version that would have the same version build number as some already released version. This is intended to avoid having different ZIP packages both appearing as the same version which effectively breaks other mechanisms in Moodle such as available update notifications.
  • When attempting to release a new version that has the version number lower than some other already existing plugin version supporting the same Moodle version, a validation warning is raised. Without further action (such as hiding the version that had existed), the newly added version would have no effect on Moodle branches supported by both plugin versions.

API for maintainers

Plugins directory has some of its functionality exposed via web service now. This allows maintainers to release new versions of their plugins automatically - for example using local shell scripts - without the need to fill the form at the web UI.

Please see Plugins directory API docs page for details.

Release by tagging at GitHub

The new API allows to implement integrations with your CI and release workflows. An example of such an integration - ability to release new plugin versions simply by tagging them at GitHub - has been published. Please see https://github.com/moodlehq/moodle-plugin-release for details.

Many thanks everybody for your valuable inputs and suggestions in relevant tracker issues MDLSITE-4781 and MDLSITE-4041.

平均分：Useful (14)
回复David Mudrák

Re: Improved validation, API for maintainers and ability to release by tagging at GitHub

Dan Marsden -
Core developers的头像 Particularly helpful Moodlers的头像 Peer reviewers的头像 Plugin developers的头像 Plugins guardians的头像 Testers的头像 Translators的头像
This is awesome David! - I can see this being really useful in keeping the versions in the moodle.org plugins db up to date - thanks heaps!
平均分： -
回复David Mudrák

Re: Improved validation, API for maintainers and ability to release by tagging at GitHub

Tim Hunt -
Core developers的头像 Documentation writers的头像 Particularly helpful Moodlers的头像 Peer reviewers的头像 Plugin developers的头像
Thank you very much David 微笑
平均分： -
回复David Mudrák

Re: Improved validation, API for maintainers and ability to release by tagging at GitHub

Dan Marsden -
Core developers的头像 Particularly helpful Moodlers的头像 Peer reviewers的头像 Plugin developers的头像 Plugins guardians的头像 Testers的头像 Translators的头像
Hi David,

I've just been looking at how github manages secrets and how it seems that it is possible for a contributor to expose a github secret by crafting a pull request. 伤心

I see the Moodle "token" for this webservice is also exposed under preferences -> "security keys" so it might be worth modifying the docs to encourage github users to hard-code a token rather than their full username/password.

I was also wondering if we could have some form of flag that allows a developer to tell the plugins db
* Only accept a download url (don't allow the zip to be included in the POST)
* Only accept a download url that matches the source control url as a base.

That way if for some reason the token is exposed it signficantly limits what can be done with it?

any thoughts?
平均分：Useful (1)
回复Dan Marsden

Re: Improved validation, API for maintainers and ability to release by tagging at GitHub

David Mudrák -
Core developers的头像 Documentation writers的头像 Moodle HQ的头像 Particularly helpful Moodlers的头像 Peer reviewers的头像 Plugin developers的头像 Plugins guardians的头像 Testers的头像 Translators的头像

Many thanks Dan for looking at it and raising these important issues.

possible for a contributor to expose a github secret

Can you share some links to resources about this? I live in impression that the issue has been fixed recently by GitHub.

to encourage github users to hard-code a token

Yes, we can make it as an option (to specify token directly). The main reason why I did not go this way as the primary one was that these tokens eventually expire (after $CFG->tokenduration which are 2 weeks on moodle.org and 12 weeks by default), which would break automation and potentially take a while for the maintainer to debug and realise what's going on.

However I share the worries of having the login credentials stored somewhere so this is definitely an improvement to be made. May I ask for creating a new issue at https://github.com/moodlehq/moodle-plugin-release

if we could have some form of flag that allows a developer to tell the plugins db

I like these ideas, thanks! MDLSITE issue reported will be appreciated.

平均分： -
回复David Mudrák

Re: Improved validation, API for maintainers and ability to release by tagging at GitHub

David Mudrák -
Core developers的头像 Documentation writers的头像 Moodle HQ的头像 Particularly helpful Moodlers的头像 Peer reviewers的头像 Plugin developers的头像 Plugins guardians的头像 Testers的头像 Translators的头像

these tokens eventually expire

One option might be that instead of self-creating the token, maintainers could ask us (moodle.org admins) to generate the token for this service manually, and set its expiration prolonged (or even without expiration). They could then have this token stored instead of username + password. If there were a big demand for this, we could even create a small UI inside the plugins directory for generating this special token.

平均分： -
回复David Mudrák

Re: Improved validation, API for maintainers and ability to release by tagging at GitHub

Dan Marsden -
Core developers的头像 Particularly helpful Moodlers的头像 Peer reviewers的头像 Plugin developers的头像 Plugins guardians的头像 Testers的头像 Translators的头像
Thanks David - when a first-time contributor submits a pull request, github waits for the repository manitainer to "approve" the running of the workflow, but on subsequent runs, it will run the actions - I haven't tested this, but after their first submission - couldn't someone modify the github action config to make a post call to their own site and include the secret at the same time, then submit a pull request with that change, which would then trigger the github action config?

Also - I made a pull request to an external plugin today and saw this setting available to me in the pull request:
pull request checkbox from github.
That seems to suggest that if ticked - someone could add a commit to my own local repo, triggering "my" actions and possibly retrieving any secrets stored in that repo - so if I have a global "secret" stored at my account level, rather than within the repo which is how I'd want to do it when managing multiple plugins, and then I create a fork of some other plugin and submit a pull request - if that box was ticked - the maintainers of the repository I have submitted to may be able to access "my" secrets?

Of course - I could be missing something there and may not understand completely how the secrets work - I haven't done much more research on it.
平均分： -
回复Dan Marsden

Re: Improved validation, API for maintainers and ability to release by tagging at GitHub

David Mudrák -
Core developers的头像 Documentation writers的头像 Moodle HQ的头像 Particularly helpful Moodlers的头像 Peer reviewers的头像 Plugin developers的头像 Plugins guardians的头像 Testers的头像 Translators的头像

Thanks Dan.

The workflow has been designed to be triggered by pushing new tags, or manually. Tags cannot be part of pull requests. So only repo owners (or better users who already have write access) can push tags. Similarly, only users and apps with the actions:write permission can trigger the workflow manually. So I cannot see how contributors could amend and trigger the workflow.

global "secret" stored at my account level

As far as I know, GitHub does not support account level secrets. Secrets are stored on repository level or organisation level.

Again, I may be wrong and miss some bit. But I tend to believe the workflow was designed with respect to GitHub's security guidelines. I still plan to re-implement it though so that only token would be used, not the login credentials.

平均分： -
回复David Mudrák

Re: Improved validation, API for maintainers and ability to release by tagging at GitHub

Dan Marsden -
Core developers的头像 Particularly helpful Moodlers的头像 Peer reviewers的头像 Plugin developers的头像 Plugins guardians的头像 Testers的头像 Translators的头像
Thanks David - your example uses tags, but we were looking at using other methods for our plugins as we don't usually manage releases using tags and wanted to go for something slightly simpler.. I was thinking we'd trigger it based on a version bump.

I just dug into this a bit more on github and found this:
"Secrets are not passed to workflows that are triggered by a pull request from a fork."
Which is a huge relief and solves the main security concern I had.

Another challenge we face - Catalyst's github org (where we store our open-source Moodle plugins, but also code from many of our other teams) has over 100 staff with contributor access, with 378 repositories (not just Moodle related ones). With the large number of Moodle plugins we support, it's more efficient for us to store the secret at the "org" level rather than in each plugin, however some of these repositories allow for "external" contributors (non-catalyst staff) who would potentially have access to expose the secrets via a custom github action. We could implement some internal policies/processes to prevent this, but I'd feel a lot more comfortable if we could store a token that was heavily restricted in what it could actually do compared with a secret that stored a moodle.org username/password.

But - for my own "personal" github repositories - using a username/password looks safe enough.
平均分： -
回复David Mudrák

Re: Improved validation, API for maintainers and ability to release by tagging at GitHub

David Mudrák -
Core developers的头像 Documentation writers的头像 Moodle HQ的头像 Particularly helpful Moodlers的头像 Peer reviewers的头像 Plugin developers的头像 Plugins guardians的头像 Testers的头像 Translators的头像

Just a follow up on the useful discussion here.

A new version of the workflow template was published today with a few improvements including the authentication security concerns. The new workflow no longer needs moodle.org username and password stored anywhere. Instead, developers are expected to generate the token directly via the web interface in the plugins directory and use it instead. This token has the scope limited to this particular service only and is therefore much safer to be worked with.

For the full list of improvements, see the CHANGES.md file at https://github.com/moodlehq/moodle-plugin-release

Many thanks to all early adopters for your useful feedback!

平均分：Useful (1)