I (honestly) hate to whinge because I know its not something we pay for...but it seems that some plugins get assessed earlier than others.
I have two that have been in the queue over 2 months now and no action whatsoever. I would just whinge and bear it. But others have arrived later and been assessed earlier in that time. I think in the past I have actually been the beneficiary of a speedy transit.
Is it possible to get some sort of indicator, even if its what place in the queue, of how close we are?
The Plugins review team is free to cherry-pick plugins from the queue but we try to review them in the basic order they come in. There should be some stats in the review queue that you can see - particularly the longest waiting plugin will give you an idea of how long it is currently taking to clear the queue.
Sometimes I only have 5 or 10min free (often while I'm waiting for a build process to end) and I might jump into the plugins db and take a look. Typically we'll do some form of "pre-flight" checks on the newly submitted plugins, and occasionally they are written so well, or so quick to review and test (like one recently submitted by Howard Miller) - they get through the queue in speedy time. Others that require a bit more effort are less likely to be picked up early and will just have to wait until they get near the top of our queue. There are also quite a few plugins in the queue from various "known" developers and the queue seems to be growing faster than we have been able to clear it over the past few months.
There are 51 plugins in our review and approx 15 that have been waiting longer than you for their plugins to be reviewed. I did a couple last week that had been waiting over 150 days but the longest waits in the queue at the moment are 101 days for an initial review and 124 days for a plugin that was resubmitted for review.
Hopefully we'll get to them soon!
Thanks for the reply. I figured it was the "what can I get done before my next meeting" approach to queue management. Thats a kind of efficiency and I have no complaints about that.
But it highlights that plugin approvals are what gets done by busy people between more pressing matters. ie they are low priority. I won't make any suggestions, because they would all involve someone allocating more resources. But I don't think anyone wants to wait 150 days.
yep.. we are always keen for more people to review plugins... if you're offering to volunteer - drop David Mudrak a message...
I won't make any suggestions, because they would all involve someone allocating more resources. But I don't think anyone wants to wait 150 days.
No, you're right. And it's one thing that particularly worries me a lot as I still feel somewhat attached to and responsible (and ashamed) for how things are in the Plugins directory. The plugins approval review process is actually something we should talk about and come up with a solution. Because the current state is not sustainable.
To put things into a wider context, it should be clarified that officially, HQ developers are not obliged to provide plugins approval reviews. They were kindly asked to do so and many do, as much as their time allows them to help. But as everywhere else, there are other projects, duties and patches reviews with higher priority (the moodle peer-review queue is even longer than the plugins approval one).
As far as I know, it's been envisioned that the plugins directory would be kind of self-maintained by the dev community itself, and the plugins guardians program was a step towards that goal. It worked, albeit maybe not to the full expected extend. And as of today, out of all guardians, it's effectively only Dan Marsden who pushes the boulder tirelessly (?) forward.
My overall feeling is that the entry standard evaluation bar has been set too high, given the lack of human power to provide such reviews. For sure, the approval reviews help to keep the quality of published plugins on a high level and from that perspective, I believe that things improved significantly. I am just not sure if the price for such a quality is adequate. It is not trivial to provide good feedback on day to day basis and the risk of burnout is high.
I'll be happy to hear the community's suggestions on how things could flow more smoothly. We have a wide range of alternatives including extreme cases like zero approval reviews. I am serious. There are many software repositories out there with very liberal approach, where any package can be published by anybody. And apparently they work and grow, too. Or, we could change the rules in more creative ways - e.g. automatically publish all submitted plugins but then have some levels of how well the plugin is perceived and used in production. Based on users feedback, ratings, stats and/or code reviews etc.
I think it's time to re-think what we (as community developers and moodle users) really need and what the plugins directory can do to support such needs better.
I'm for the attitude of 'help you help me'. So as there is a length of time associated with a plugin, then to get priority the developer should provide evidence of meeting the coding standard, tests etc. Even using automated tools like Travis CI where various different checks have been done and any 'exclusions' justified. This then brings not only credibility to a plugin but (I imagine) reduce the review time.
If a developer can't be bothered to put the work in and demonstrate that what they create is of a given quality then why should they ask someone else to find all the faults for them? For example with my 'Foundation' theme you can see that I progressively reduced the code pre-checks -> https://moodle.org/plugins/pluginversions.php?plugin=theme_foundation in the Moodle 3.5 version whilst it was in the review queue (Initially wanting to make sure I got the name I wanted when I had something I could then tidy up) and even requested a pause when I found an issue -> CONTRIB-7493.
I'm not for 'zero approval reviews' as that would degrade the perceived 'quality' of the plugins available and be detrimental to the image that Moodle has.
The volume of submissions, and quality of new plugins, surprises me. Because Moodle plugin development is not that easy, and there is usually no commercial incentive to be listed. I think that the stream of plugins coming in is a sign that at least some things are working very well. So when we look at the issue of plugin approvals backlog, I feel we should keep sight of the fact this is a problem of managing growth. (ie a good problem).
I am not in favor of zero approval reviews. I think we have a good thing going, and that might ruin it. We might speed up the process by lowering the bar, its hard for me to say how much gain we would get from that.
My suggestion is perhaps a bit predictable because I have said similar things in the past. I suggest we have a minimal approval review. It could be almost entirely automated actually. The plugin installs, uninstalls, backs up and restores.(perhaps a bit more ) ... Green tick.
Then we have a certified plugin review which costs money, and the plugin receives a far more rigorous test. The plugin then has a certified badge. This would limit the number of rigorous reviews required, and also go someway to pay for the time of the partner/HQ developer. And it would also give the applicant a transparent and predictable path to certification
I thought it might be useful to give a summary of the reviews from approx the last month.
7 Plugins were approved in the database (some of these failed an initial review but were fixed and resubmitted.)
Approx 28 plugins failed a review:
- 3 plugins were badly copied forks of other existing plugins and added some extra code with significant security issues.
- Lots of raw use of $_POST/$_GET
- PHP mysql functions instead of Moodle DB API.
- Lots of raw params injected into inline SQL.
- Invalid namespacing/table names.
- missing login checks
- missing capability checks
- hard-coded role-ids or checks for a role with the shortname "student" instead of a capability check.
- some fail to install on postgres, some use specific Mysql only functions.
- Non-GPL compatible 3rd party code included.
- Major Moodle trademark violations (significant advertising of Moodle services without being a registered Moodle Partner.)
- One plugin submission only supported Moodle 3.0 and older - it did not support a current stable supported release.
As part of the review we tend to report a lot of other stuff that we don't use for plugin rejection but encourage the developer to fix.
Obviously the failed plugins require a massive amount more effort from the reviewer - not only do we try to educate the plugin developer we also tend to provide a small amount of help as they address the issues we have reported.
I'm not a fan of moving to "automated reviews" or "no reviews" - it would be good to have some more volunteers to help with the review process though!
I do like where Daniel is going with his forum/discussion ideas though - maybe we should force plugin developers to post in the forums first, introducing the plugin and with a link to their source control repository - allowing for more community feedback prior to approval in the plugins db.
Well that's quite shocking about the nature of problems with failed plugins but I suppose not surprising. Moodle plugins are a world of their own. And it takes experience to write a "good" one.
Personally I think its a deeper issue than just Applicant's trying harder, or getting more volunteers. How important are plugins really in the Moodle world? Right now for developers reviews are voluntary and low priority. Is that ok? Is just adding "[other people] should..." to the equation really going to make a difference in the long run? What happens If the review queue gets to 200 days or 365 days? What would that say about Moodle as a choice of platform?
I think it would say that MoodleHQ does not care much about 3rd party plugins and their developers. And this is why I suggest a structural/strategic response that does require some commitment from MoodleHQ
Or, we could change the rules in more creative ways - e.g. automatically publish all submitted plugins but then have some levels of how well the plugin is perceived and used in production. Based on users feedback, ratings, stats and/or code reviews etc.
Yes, I think this should be the way.
| LEVEL | Plugins in here | Evaluators | Pass Criteria |
|---|---|---|---|
 |
Plugins in this category are of high code quality and are highly appreciated by the moodle community |
Moodle Plugin Guardians |
All current criteria |
 |
Plugins in here are highly appreciated by the moodle community. |
Community |
Likes |
 |
All new plugins |
Automated |
Basic criteria such as: Git Repository, description, etc. |
- Increase transparency because new (maybe genius) plugins would not be hidden in trackers and forums
- reduce moodle plugin guardians workload because they only need to approve those plugins which are appreciated by the community and of which the maintainer feels that they are actually ready to be checked
- leverage the community to participate in the plugin evaluation process
- motivate plugin developers to release early and grow the plugin in cooperation with the community
- make waiting more acceptable because it is not an all or nothing approach. If I as a maintainer have my plugin in the red or yellow category, it does not matter so much when I finally move up to the green level. But if my work is totally hidden until I can get in contact with the community via the repository, each day increases frustration
The plugins database is a real gem for Moodle, and if its growing thats a fantastic thing. I think it is important to make some effort to build a process that can be maintained. Flotter's suggestion makes sense. And basically I agree.
Devils advocate says: It is quite a big change though. And somebody has to do all that work to put it into place. That probably has to be David Mudrak because the code is not open. If its just impatient developers, and not the bazillion Moodle users, then isn't this really just a "nice to have."
So while we are brainstorming, here is another idea. The plugins database has a nifty feature where tagged Github releases, as long as they are registered with the plugin, can be pulled into the "Add a new version" process. I think it would not be too difficult to add this feature to the Moodle plugins installer. Currently you can upload a zip or install from Moodle.org. How about, we add a third method? We allow the admin to paste in a github repo url. Then the plugin can be installed straight from Github, and the Moodle site can check for new releases and flag them in the same way it does for new releases in Moodle.org.
This would not involve a change to the plugins database. It would be a really nice feature. It would assist plugin authors who promote their new plugin in the Moodle forums, before being released in the plugins database. Because its a lot easier than telling rank and file how to get installed from a git hub repo. And it provides a nice way for users to get version updates.
Authors of plugins not released on Moodle.org would appreciate this too. I think many of us have plugins we have not released but are used by clients, colleagues and ourselves. So this would be great.
If there was agreement, we could even have listings for such plugins in the plugins database that show up in searches, but are not approved, or distributed from there.
You can already add a new version from GitHub. When specifying the file to upload, just select 'add', and choose 'github', put in your GitHub username (remembered if you have done this before) and select the repo. It doesn't automatically poll for new versions, but it's a reasonable compromise and is pretty quick to do.
Hi Davo, I think that you are referring to adding a plugin to the plugins database. And yes there is a Github integration the way you describe there.
My suggestion was actually to add a feature similar to that to the install add-on tool in Moodle itself. Like this:
I'm still heavily against removing an initial manual check by a developer prior to approval in the plugins db (although anything we can do to improve this or potentially make the process more transparent is good.)
To give some context - we have done pretty well clearing the backlog and there are now 27 plugins in the review queue, most plugins have been waiting for less than 60 days. We also have at least one new plugin reviewer coming on board and hopefully more of you reading here might volunteer to help! - Even the Peer review queue in the Moodle Tracker is longer than 27 issues!
One example of why I am a strong advocate for a manual review process - In this past couple of weeks, I've rejected 2 plugins which had taken another 3rd party theme, renamed it and replaced all copyright headers with their own name, had not referenced the original theme developer anywhere and had corresponding "marketing" on their own websites claiming the themes were their own work.
In another example, I've reported some serious security issues with a plugin, given reasons why it is an issue, and pointed at relevant documentation to help them understand what to do (I have even given code examples of the right way to do things...) the developer has told me 2 times they have "fixed" the issue but has not - (I have now reviewed it 3 times).
Most developers are pretty responsive and appreciative of the code review we provide - it's often a really good learning experience, but for other developers (particularly integrations with "commercial services") getting their plugin published in the plugins db is a really high motivator for them to fix the issues we report during the review.
If people want to share their new plugin before it get's approved in the plugins db there is nothing stopping them from creating a post in the forums with a link to their source control repository and asking for feedback - many people already do this
We track the plugin reviews using the Moodle tracker and the details of rejections are usually posted as issues in the developers tracker but also summarised in the tracker issue itself. Interested parties might like to dig around in the tracker issues that are flagged for "changes required" to see why plugins are being rejected - the tracker doesn't always provide a direct link to their issues tracker, but 99% will use github or bitbucket and you should be able to just search those sites for a plugin with the same name to find the actual code.
Man thanks Dan for sharing this.
But if my work is totally hidden until I can get in contact with the community via the repository, each day increases frustration
But that is not the case. It is not "totally hidden". As Daniel Thies pointed out in this thread already, the plugin authors have ways to present their early work to early adopters. The code is available on Github and they can present and promote it via moodle.org forums, twitter channel and eventually other media.
But I get the point and it is actually in line with what was suggested at the DevJam day during the MootDE17 in Mannheim (and also before at other events I am aware of). Let us have a public list of plugins waiting for approval with just the plugin name, maintainer, Git repository link and eventually the queuing time. This way, we can give the early adopters a chance to preview the coming-soon plugins early, without lowering our quality standards.
Good suggestion.
Yes such a list would work.
Just a point about advertising your plugin on the forums before its released. I generally do this. Sometimes there ends up being a good conversation there. Which is great. But when the plugin is released it really needs a new forum thread because the original post is talking about github and zips and beta things. And you can not edit that. But conversations attract traffic and you end up having conversations in both old and new threads.
A good example is the Generico one here ...
https://moodle.org/mod/forum/discuss.php?d=261896
Its more of a cautionary tale, than a problem. Probably the best thing to do is to ask Helen to lock the conversation and make the final post a pointer to the new one.
> Let us have a public list of plugins waiting for approval with just the plugin name, maintainer, Git repository link and eventually the queuing time.
I really like this proposal.
Currently, I am watching https://tracker.moodle.org/issues/?jql=project%20%3D%20CONTRIB%20AND%20status%20%3D%20New%20AND%20component%20%3D%20%22Plugins%20reviews%22 regularly, but it is quite cumbersome to have to search for the plugin code on Github or even on the open web if a plugin's name or frankenstyle name sounds interesting but there isn't any information where to find it yet in the plugin approval issue.
I really like this proposal.
I am sure you do. You suggested it, among all the other good suggestions, in Mannheim
> I am sure you do. You suggested it, among all the other good suggestions, in Mannheim
I just picked a more polite phrase than "Yeah, that was my proposal two years ago - why didn't you build it yet?"
Ok, it took a while. But here it goes - https://moodle.org/plugins/report/index.php?report=unapproved_plugins_public
Hope it helps. Thanks again for the reminder and patience!
But that is not the case. It is not "totally hidden". As Daniel Thies pointed out in this thread already, the plugin authors have ways to present their early work to early adopters. The code is available on Github and they can present and promote it via moodle.org forums, twitter channel and eventually other media.
Of course it is possible to put a plugin on Github or anywhere else on the web and promote it via the moodle org forums. But in the moodle.org repository, they are hidden to the public until approved.
But I think it might also be interesting to open up the whole thing a bit and foster early cooperation between plugin developers and the community of users with the help of the plugin db. Plugin developers could upload their plugins in an earlier stage. Moodle admins would find those plugins early and could influence and support development. Already now, downloads would be counted and likes collected. Only after some time (ie, when ready), the plugin would enter into an approval process.
A greatly underestimated aspect of the process is the usefulness of the the general discussion forums. Developers should not rely entirely on the database and the guardians for plugin review. If a plugin is of interest to the community, it should be easy to get get a few testers, corrections, suggestions, and maybe even a brief initial code review by announcing it clearly in an appropriate forum. Once a plugin is announced in the forums, it is already available to motivated users. If a plugin does not receive useful feedback in the forums, maybe it is not of sufficient interest to submitted to the database.
A forum discussion is part of the submission checklist. Unfortunately, it can easily viewed as an afterthought rather than a real prerequisite to the process.
One of the interesting things about this thread is that every contributor so far is a very significant contributor to the Moodle project (Moodle dev deities in my view, check out their profiles) Moodle without plugins is not really Moodle. If you disagree with that statement tell me about the real world significant Moodle installations that use no plugins.
So what is to be done?
More promotion (ads on Moodle.org?) of paid for development resources, e.g. lets not be afraid to say good resources should cost money for a GPL software solution, e.g.
The excellent dev book by Tomasz https://leanpub.com/moodle35
The online course https://www.moodlebites.com/mod/page/view.php?id=19542 which Justin Hunt has taught on.
Loving and fussing from HQ to developers, I believe we developers are as open to flattery as much as the next person. Interviews and profiles merchandise, badges etc.
Resources put into free developer stuff, e.g. the wiki tutorial stuff. Some liaison with Universities that might give their students experience developing Moodle code.
It would be nice to have an estimated date to put the final line of YUI out of its/our misery
So what is to be done?
Well... let me just dump few ideas here as part of the brainstorm.
One idea is to make Moodle development "cool" and "modern". I would like to be able to say to new wanna-be web developers to say: look, learn how to code Moodle and contribute to the project. That will help you learn the skills you will need for real-world development. What you will learn here, you will be able to apply to other web projects in the future.
Few ideas on how to improve:
Implement standard naming convention
- PSR1 and PSR2
- change variable names from moodlecase to either camelCase or snake_case
Implement more OO in mOOdle.
Some easy wins are obvious classes that we should have like User or Course.
Implement object-oriented layer for the HTTP specification by using Symfony's component: https://symfony.com/doc/current/components/http_foundation.html
Replace custom Database Abstraction Layer with Doctrine DBAL
This
is not about re-writing whole DB API, just about DB Abstraction Layer.
Do we really need a cutom one in Moodle? Doctrine DBAL http://www.doctrine-project.org/projects/dbal.html is an active, stable project that support many more DBs than we do and provides many more features.
Implement MVC
Follow Symfony & Drupal and use https://symfony.com/doc/current/components/http_kernel.html .
Use composer
Use composer & vendor directory for importing 3-rd party libraries (instead of copying them into lib). It's basically to move one and implement https://tracker.moodle.org/browse/MDL-49672 .
Replace Moodle Forms API with modern library
Moodle's forms are basend on an ancient library, let's replace it with Symfony's form component: https://symfony.com/doc/current/components/form.html (yes, I do know how much work it means)
Re-implement Moodle plugins as composer plugins.
I have no experience there whatsoever but that should be possible - https://getcomposer.org/doc/articles/plugins.md.
Standardize Moodle CLI
Based on my recent research, the most popular and active CLI library is Symfony console component - http://symfony.com/doc/current/components/console.html. Let's create Moodle CLI scripts based on that.
Implement proper logging
Implement modern logging, possibly based on monolog. This is Moodle issue https://tracker.moodle.org/browse/MDL-50592.
Replace mustache with twig
Seems like mustache for PHP is dead (https://github.com/bobthecow/mustache.php/tree/master last commit on Jul 11, 2017) and twig is a way to go.
Get rid of YUI completely
What problem would that solve?
"Use composer"I find composer is an a sequence of incomprehensible incantations I type untilI can ignore it again for a few weeks/months. That may say more about me than about composer.