Support Composer dependency manager

Re: Support Composer dependency manager

by Tim Hunt -
Number of replies: 16
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

I'm not yet convinced by this.

I'm not sure i want to pollute my plugin with an extra file, if it only benefits a few people. (Could the composer file not go in the db folder? How does composer know when there is a new version, etc?)

Moodle has its own built-in plugin-installation system for people who want easy plugin install, and that checks things like Moodle version compatibility.

In reply to Tim Hunt

Re: Support Composer dependency manager

by Brendan Anderson -

Maybe an individual plugin developer should not, but I think Moodle as a community should. The reason we should is because the built in plugin installation system is inadequate for automation. As developers we need a way to automatically bring up new development environments and deploy changes.

I'm not sure i want to pollute my plugin with an extra file, if it only benefits a few people. (Could the composer file not go in the db folder?

The people looking at the actual files in the plugin are developers who are familiar with composer.json files. They will expect to find it in the plugin root folder. Right now, not many people can take advantage, so you are right that it will only benefit a few people. But imagine if this was the standard for Moodle. We could all develop more quickly and deploy more reliably. That's a big win.

How does composer know when there is a new version, etc?

Composer knows when there is a new version because you tell packagist.org that there is a new version. This can be easily automated with GitHub.

I am still fairly new to Moodle, but I have been really surprised with the lack of interest in this type of activity. If I bring another developer onto my Moodle project and tell him that he needs to manual download all the plugins I use, or worse, checkout a monolithic VCS repo with nothing but code I didn't write, I will get straight up laughed at.

In reply to Brendan Anderson

Re: Support Composer dependency manager

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

You have one background, and one set of assumptions, but other people know different things. E.g. I am a developer who is always looking at file in my plugins, but I have no clue about these composer.json files. Also, I have no desire to tell packagist.org about my plugins. They are all in moodle.org/plugins. (If you were interested to build an automated link from the Moodle plugins directory to some other repository system, well talk to David Mudrak, but whatever you do should require zero additional work for plugin maintainers.

Co-incidentally, yesterday just before I saw your other thread, I had just finished reading this blog post: http://www.vitavonni.de/blog/201503/2015031201-the-sad-state-of-sysadmin-in-the-age-of-containers.html wink

Average of ratings: Useful (2)
In reply to Tim Hunt

Re: Support Composer dependency manager

by Darko Miletić -

Docker is a great thing but it requires different mindset and will to try something new. Having an automated cli way to download and install Moodle plugins would also help automating stuff. With that implemented writing a docker script for whatever kind of setup would be fairly trivial.


Average of ratings: Useful (1)
In reply to Tim Hunt

Re: Support Composer dependency manager

by Brendan Anderson -

You are right. Composer may be new to Moodle developers. And now the moodle plugin directory exposes an API for getting information about plugins in an easy to parse way (so we don't really need packagist). I am not married to composer. Composer is a tool for solving a similar problem, but not the exact problem we have. After all we are not autoloading anything, we just want to download the files.

And composer will introduce extra work for plugin maintainers.

I think it is a viable option because it already exists and is the standard for dependency management in PHP.

But Drupal has been facilitating automation without it using drush make. It's a concept that I think can work for us, the idea being that the core version and plugin versions are defined in a configuration file which is parsed when the command is run and the project is built.

In reply to Brendan Anderson

Re: Support Composer dependency manager

by Just H -

"But Drupal has been facilitating automation without it using drush make."

Haven't looked into it but there's a similar plugin for Moodle inspired by Drush; see Moosh.

In reply to Just H

Re: Support Composer dependency manager

by Daniel Neis Araujo -
Picture of Core developers Picture of Plugin developers Picture of Translators

Just to note, Moosh uses composer to generate new plugins from templates that are on github and have composer.json files =)

And although you can use Moosh to install plugins for you,

there isn't a way (at least i haven't found) to have a file

like composer.json telling everything you want and just

install eveyrthing with one click.

For what i know, Moosh only let's you install plugins one by one.

I think "you can have a script to do that" is not a reasonable answer.

I've used git repo and git-submodules in the past to try to handle better my colletion of plugins

acrros different Moodle deploys, but composer seems to be better,

even if you need to add a composer.json file or even clone the original code

to your repo to have this file on it while you waiting for the original maitainer

to get convinced of the benefits ;)

FInally, in "container" and "virtualization" times, I've asked people of digital ocean to offer moodle as 1-click-app : https://digitalocean.uservoice.com/forums/136585-digitalocean/suggestions/7606407-offer-moodle-lms-as-1-click-app
And there is a docker image for Moodle (https://github.com/playlyfe/docker-moodle) although it not worked on my debian-jessie with docker 1.3.3 , but fortunately is my fault and not a bug ;D

Kind regards,

Daniel

In reply to Brendan Anderson

Re: Support Composer dependency manager

by Jay Knight -

I wonder if there could be a composer-like way to enumerate third-party plugins that you'd like to install in your config.php file, and have the installation/upgrade scripts install them and keep them up to date.  This would be a nice way to keep test/dev/prod deployments how you want them, without having to manually install plugins either via the web interface (I keep that disabled so other admins don't go trying to install stuff) or unzipping things in the right directories and confusing git*.

* As a bonus, it sure would be nice for third-party stuff to not be strewn about in various git-tracked subdirectories (mod/auth/block, etc). A configuration option to point to a directory of plugins outside of the moodle web root would be nice, but I understand that would require lots of changes to how modules work.

In reply to Jay Knight

Re: Support Composer dependency manager

by Daniel Neis Araujo -
Picture of Core developers Picture of Plugin developers Picture of Translators

Hello,


just to note, mod Scheduler

https://moodle.org/plugins/view.php?plugin=mod_scheduler


have just merged the code adding composer.json

https://github.com/bostelm/moodle-mod_scheduler/pull/21


=)

In reply to Daniel Neis Araujo

Re: Support Composer dependency manager

by Henning Bostelmann -
Picture of Core developers Picture of Plugin developers

...and I have removed the file from mod_scheduler again.

Sorry, but when I saw the pull request, I was (foolishly) under the impression that these files were part of some agreed recent API change and were being added to all plugins. As it turns out, this was not the case and the debate is still ongoing. I find these tactics of "pull request first, discuss later" rather questionable.

Henning, mod_scheduler maintainer


In reply to Henning Bostelmann

Re: Support Composer dependency manager

by Juho Jaakkola -

There is no "debate". You either:

  • Support composer and make life easier for developers and sysadmins
  • Don't support it and make their life more difficult

It's as simple as that. What is there to debate about?

People in this thread have used a lot of time and effort trying to explain why it would be a very positive thing if Moodle and it's plugins supported composer. I don't understand why their effort is being neglected.

Average of ratings: Useful (1)
In reply to Juho Jaakkola

Re: Support Composer dependency manager

by Petr Skoda -
Picture of Core developers Picture of Documentation writers Picture of Peer reviewers Picture of Plugin developers
To me it seems that composer is not the ultimate solution, but instead just several new problems for core developers. I do not think there is any simple solutions here because contributors seem to not have much motivation to keep up with all the API changes and new versions. Moodle was not designed to work with composer.

I have been personally in several positions working on Moodle codebase, and my personal opinion is: Please not another new versioning and related APIs!
In reply to Petr Skoda

Re: Support Composer dependency manager

by Juho Jaakkola -
it seems that composer is [...] just several new problems for core developers

What are the problems? Could you make a list of them? I'd like to take on the challenge to try to find a solution to each one of them!

Please not another new versioning

Composer does not require you to change versioning. You can use existing git tags or even individual git commit hashes to define the version you want to use.

Please not another new [...] APIs!

Composer does not require you to add new APIs.

Average of ratings: Useful (2)
In reply to Juho Jaakkola

Re: Support Composer dependency manager

by Daniel Neis Araujo -
Picture of Core developers Picture of Plugin developers Picture of Translators

Hello, everybody


I also can't see real problems for developers. If you are a new developer (or an old one) you already have to use "grunt" and setup things for it to work.

If you write and/or run tests, you also use composer to install the dependencies.


Also, besides all the other things already said about how composer can help Moodle, there is two more worth mention:

"Update Notifications

To receive notifications about new version releases you can sign up for VersionEye, a web service that can monitor your GitHub and BitBucket accounts for composer.json files and send emails with new package releases.

Checking your dependencies for security issues

The Security Advisories Checker is a web service and a command-line tool, both will examine your composer.lock file and tell you if you need to update any of your dependencies."

http://www.phptherightway.com/#composer_and_packagist

This is not just for plugin developers but also for Moodle own dependencies (https://docs.moodle.org/dev/Moodle_libraries_credits). If instead of relying on a wiki page to document the used libraries, we could use the composer.json file (the history will be on git) and use composer to update libraries when needed and also use the above services to help us know when such update is needed.

Here are the links to libraries used by Moodle that are on packagist (same order as link above):


Considering that we have 30 dependencies (and some are javascript libraries, at least one is "Spike PHPCoverage" that has an outdated link) and we have already 13 here (half of libraries),  maybe composer is still not the ultimate solution, but it is getting there. All these people would not use it if it was just "new problems and versioning". Also, we could start thinking about the quality and future of libraries that do not adhere to these new trends.

You may also like to take a look at http://thephpleague.com/

Kind regards,
Daniel
Average of ratings: Useful (1)
In reply to Brendan Anderson

Re: Support Composer dependency manager

by Matt Gibson -

This is is a really good idea. The use case I'm thinking of is where the individual plugins need to pull in outside dependencies.

I've had many problems with putting libraries into plugins, which are then also included in other plugins, causing a breakage (e.g. a class is already defined). For example, having multiple versions of the PHPCAS library included by different auth plugins, or wanting to use Symfony components, which some parts of core already include, but at different versions.

Composer should be able to sort all of this out by making sure that a version of the library which works for all of the plugins which require it (and core) is added to the class autoloading path via /vendor.

This would also be a better way to include libraries in core rather than bundling the actual code (assuming the libraries are available via composer).

I would imagine that running composer during each upgrade run, after telling it where the plugin composer.json files are located should be enough to make this work. I'm going to experiment with it over the next few weeks by running composer during 

It would be trivial to have the Plugins directory include information on how to add a plugin via composer instead of downloading the code, if that was a route we wanted to support. The composer.json file can include links to a github repo, rather than packagist.org, so no extra work for plugin devs, really. https://getcomposer.org/doc/05-repositories.md#loading-a-package-from-a-vcs-repository

Average of ratings: Useful (3)
In reply to Matt Gibson

Re: Support Composer dependency manager

by Juho Jaakkola -

Matt, how did your experiments work out?

I'm completely absolutely totally in favor of making it possible to manage Moodle sites and community plugins using composer. It would remarkably reduce the development and maintenance overhead of developers and sysadmins.

And as stated - it is a PHP *standard*. You learn it once and you can use it in any PHP project.