Downloadable list of all plugins

Downloadable list of all plugins

by Phil Lello -
Number of replies: 4

Hi all,

I've started experimenting with making Moodle and plugins installable via composer. There's a small patch available on MDL-54834 to add the necessary attribute to composer.json and a proof-of-concept at https://github.com/dunlop-lello/moodle-composed.

So that I can accelerate experimentation with this approach, I'd like to set up a composer repo with automagically build (composer) packages for all the contributed plugins on https://moodle.org/plugins/. Is there any kind of API to retrieve this information? At a minimum I'd a list of VCS links (preferred) or download links for each version of each plugin; it would be useful to have title, description, author etc. too.

Phil

Average of ratings: Useful (2)
In reply to Phil Lello

Re: Downloadable list of all plugins

by Michael Aherne -
Picture of Core developers Picture of Peer reviewers Picture of Plugin developers

Hi Phil, I did a proof-of-concept composer repository for the Moodle plugin directory a while back, and it's available at http://micaherne.github.io/moodle-plugin-repo/

You can get the code that generated the repository JSON at https://github.com/micaherne/moodle-plugin-repo/. It's not particularly nice code, but should be a reasonable example of how you can do it. If there are any bits that don't make sense, just ask!

I'd be really interested to see where you go with this idea, as I do still think it's an interesting one.

Cheers, Michael

Average of ratings: -
In reply to Michael Aherne

Re: Downloadable list of all plugins

by Phil Lello -

Hi Michael,

Thanks, that looks very interesting. Did you manager to track down source for the "plugins" plugin used by moodle.org/plugins (I can't find a public repo for it), or is this reverse engineered for the plugin version check code?

Unfortunately it looks like the name/description/maintainer still isn't in the json feed from MoodleHQ, however since the ID in the json feed maps to the plugin details page (http://moodle.org/plugins/view.php?id=XXX) and the HTML is structured well enough to scrape, I'll look at adding those in.

As a short term solution, I think the next stage in my experiment is to script a download of each version of each plugin, extract it, normalize structure, automagically generate the composer.json, and upload. Longer term, I hope I can get MoodleHQ to do this process on their side and publish the artifacts.

Cheers,

Phil

Average of ratings: -
In reply to Phil Lello

Re: Downloadable list of all plugins

by Michael Aherne -
Picture of Core developers Picture of Peer reviewers Picture of Plugin developers

Glad it was useful! I don't know if the plugin directory plugin is open source - I think it might not be. If there are fields that would be useful that aren't currently in the json feed, it would definitely be worth putting an improvement suggestion into the Moodle tracker to see if they could be added (there's a component "moodle.org/plugins" in the "Moodle Community Sites" project for things like this).

I didn't realise you were looking to actually generate new versions of the plugin packages, and I must admit I'm not quite understanding what the benefit of that would be. It would be great to be able to use the information version.php, the plugin database pages and the code repo to generate a more detailed composer.json for each package, with dependencies etc, but I'm not sure why you would embed that in the plugin source rather than just put it in the packages.json repository file. Can you give an example?

Average of ratings: -
In reply to Michael Aherne

Re: Downloadable list of all plugins

by Phil Lello -

Hi Michael,

I must admit I'd been focused on the solution I'd already started working on before looking at your work, so re-packaging probably isn't necessary.

The main reasons for doing it were:

- I'd like to massage the plugin standard to require composer.json files, and thought repackaging to include them would be a good way of bootstrapping change. Certainly creating patches for the plugins that include repo metadata is on my wish-list - I'm always a little hesitant to rely on externally generated metadata for a project, as synchronisation strikes me as a point of failure.

- I'd like the vendor/... directories to have a common structure. That said, with my latest approach to building the Moodle directory, it probably becomes more about aesthetics than achitecture, as instead of symlinks from the docroot to vendor/... I'm moving directories and symlinking vendor/... to locations under docroot. The use case that inspired this for suites where the zip starts from docroot and has sub-plugins in the appropriate locations - MediaCore did this, as do Kaltura.

Will hopefully push some updated code this weekend, as this is a spare-time/on-brand project rather than client-funded.

Phil

Average of ratings: -