RFC: subplugins.php conversion to JSON

RFC: subplugins.php conversion to JSON

by Andrew Lyons -
Number of replies: 10
Picture of Core developers Picture of Moodle HQ Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers Picture of Testers

Hi all.

As part of a set of changes which we are currently looking to integrate, we are considering making some changes to the way in which core defines its list of plugintypes and subsystems, and the way in which plugins define their subplugins.

These changes are discussed in MDL-62497.

Currently, if a plugin supports subplugins (e.g. mod_assign, mod_book, mod_lti, etc.) then it must define a db/subplugins.php file which specifies the prefix of the subplugin, and its path.

We are looking at moving this to a JSON structure in Moodle 3.7 such that it can be read by parts of our tooling process that are not able to use PHP.

We are also looking at moving the hard-coded list of plugin types, and subsytems to a similar JSON structure in lib/components.json.

Exact details are still to be confirmed.

Why can't you just use PHP?

Much of our Continuous Integration system makes use of Docker.

Our PHP images do not have NodeJS installed, and our Node images do not have PHP installed.

In addition to this, linking Node and PHP sets a potentially difficult precedent which we feel we should avoid.

Can't you just install Node in your PHP images?

Technically yes, but one version of PHP may support as many as five versions of Moodle, whilst one version of Moodle is only intended to be built with one major version of NodeJS.

We need to be able to switch versions of Node in-line with Moodle, but that may not tie to a specific version of PHP.

Whilst we could use NVM, this further complicates the build processes.

But what about my plugin..?

If you have a plugin, and that plugin defines a set of subplugins, then we will be encouraging you to move the db/subplugins.php file to a JSON structure.

We are aware that it's pretty common to have a single version of a plugin used by many versions of Moodle and we intend to provide a recommended compatability version of te subplugins.php which loads its data from the JSON structure.

This will allow you to have one authoritative structure used.

That sounds cool... can I use it?

Yes!

You'll be able to use these subplugins.json files, and the core lib/components.json if you have specific needs - for example in Continuous Integration tooling.

Can't you keep the PHP source authoritative and just build this?

In theory we could do so, and we did originally propose to do so, but then we need to have PHP available to NodeJS, and we encounter the limitations described above. Alternatively we need to add a build step whereby you generate an up-to-date copy of the cached structure as part of your build process... which is a more frustrating task than just doing it as a one-off change.

Questions..?

If you have any questiosn, please ask me.

We will update the documentation as part of this change.

Andrew

Average of ratings: Useful (3)
In reply to Andrew Lyons

Re: RFC: subplugins.php conversion to JSON

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

As someone who mostly just writes PHP code within Moodle, my immediately reaction was "this ain't broke, so what are you fixing". However, once Andrew explained all the things that would be easier with JSON than PHP, this makes perfect sense to me. I don't think the change will be that much work for plugin maintainers - and as it says in Andrew's post, they have considered that.

Average of ratings: Useful (1)
In reply to Tim Hunt

Re: RFC: subplugins.php conversion to JSON

by Marcus Green -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers

One of the things I took away was that node/nvm/npm would not be mandatory. I find those tools to involve a sequence of random incomprehensible commands that eventually work for no discernible reason and then I forget until the next time I urgently need something I didn't know I needed. But perhaps that is just me.

Average of ratings: Useful (1)
In reply to Marcus Green

Re: RFC: subplugins.php conversion to JSON

by Andrew Lyons -
Picture of Core developers Picture of Moodle HQ Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers Picture of Testers

Hi Marcus,

We originally raised the proposal to support ES6 in these forums back in July.

Using Grunt (and therefore node/npm) has been mandatory for any JS development, as well as any use of the clean theme for several years now.

The good news is that with these changes and those discussed in MDL-62497, we're actually making the process simpler going forward. Previously in order to install the toolset it was a case of running `node install && node install -g grunt` and to build your AMD-based JS the command was `grunt amd`. For clean it was `grunt less`.

Now the full command set is `node install && node install -g grunt` to install, and then `grunt watch` for everything else, and it will watch for any changes. You can also still issue individual commands.

Which commands is that you're unsure about?

Andrew

Average of ratings: Useful (1)
In reply to Andrew Lyons

Re: RFC: subplugins.php conversion to JSON

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

Well, when the commands you mention in your post work, it really is that simple.

However, they don't always work. And when they fail you just have a monterous black box, and only very cryptic error messages to go on.

Average of ratings: Useful (3)
In reply to Andrew Lyons

Re: RFC: subplugins.php conversion to JSON

by Marcus Green -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers

"Using Grunt (and therefore node/npm) has been mandatory for any JS development, as well as any use of the clean theme for several years now."

I used all my cunning to avoid it until about 6 months ago  and also kept my plugins working with the latest versions of Moodle. This does not make me a good person smile


Average of ratings: Useful (1)
In reply to Andrew Lyons

Re: RFC: subplugins.php conversion to JSON

by Justin Hunt -
Picture of Particularly helpful Moodlers Picture of Plugin developers
Hi Andrew


Once I am all set up with Grunt and the right version of node (always always use nvm) I am perfectly terrified of touching it. Thats not my main point.

Keeping non PHP systems and php ones as distinct as possible in the build process, certainly seems the more robust way. And if that can all be done by using a JSON descriptor file, yay. I guess the devil is in the details though. 

For average joe/jane plugin developers who do not make "plugins with subplugins", but who do make and maintain "subplugins"  ... How would all this affect us?

Average of ratings: -
In reply to Justin Hunt

Re: RFC: subplugins.php conversion to JSON

by Andrew Lyons -
Picture of Core developers Picture of Moodle HQ Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers Picture of Testers

Hi Justin,

This particular change will not affect you at all.

As I say, the move to a JSON format for the subplugin configuration file only affects a very small number of people.

Andrew

Average of ratings: Useful (1)
In reply to Andrew Lyons

Re: RFC: subplugins.php conversion to JSON

by Darko Miletić -

If I understand you correctly when you say:

but one version of PHP may support as many as five versions of Moodle, whilst one version of Moodle is only intended to be built with one major version of NodeJS.

That implies that there is strictly defined relationship Moodle version - NodeJS version. Is that documented somewhere?




Average of ratings: -
In reply to Darko Miletić

Re: RFC: subplugins.php conversion to JSON

by Darko Miletić -

Looking at package.json shipped with Moodle following can be deduced:

Moodle 2.9 - grunt 0.4.5, node  (implicitly 8.x)
Moodle 3.0 - grunt 0.4.5, node  (implicitly 8.x)
Moodle 3.1 - grunt 0.4.5, node >= 8.9, < 9
Moodle 3.2 - grunt 1.0.1, node = 8.9
Moodle 3.3 - grunt 1.0.1, node >= 8.9, <9
Moodle 3.4 - grunt 1.0.1, node >= 8.9, <9
Moodle 3.5 - grunt 1.0.1, node >= 8.9, <9
Moodle 3.6 - grunt 1.0.1, node >= 8.9, <9

So we can say that all versions of Moodle that work with grunt support Node 8.9+ (lts/carbon). On Debian based distros this can be installed like this:

apt-get -qq update
apt-get -qq --no-install-recommends install curl ca-certificates git-core gnupg
curl -sS https://raw.githubusercontent.com/creationix/nvm/v0.34.0/install.sh | bash
. ~/.bashrc
nvm install --no-progress lts/carbon


Not really sure where is the complication.







Average of ratings: -
In reply to Darko Miletić

Re: RFC: subplugins.php conversion to JSON

by Darko Miletić -

And just to add on all current versions that are taken into account for CI testing you might need to add these packages:

npm install --no-progress -g grunt@1.0.1
npm install --no-progress -g grunt-cli@1.0.1




Average of ratings: -