Just a thought here > I realise we have a rating system for each module & plugin in the database on its respective page (but most ppl don't use it). Moreover there ISN'T a system that actually ranks or compares the credibility of the plugins. We all appreciate the time & effort developers invest to make these plugins available to our community, but lets face it - some are more stable & better maintained than others.
As a point of comparison, Joomla! CMS gives users the opportunity to favour, rate, recommend, & review extensions. Users can also browse by most favoured, top rated, most popular & reviewed.
Do we need to devise a more effective system that makes it easier for Moodle community members to make an informed decision about which plugins are(not) reliable?
Ease of installation, functionality, stability, moodle-like appearance & overall rating are some likely assessment criteria. Administrators, developers & end-users should be able to evaluate the contributions of others. On the same token, this may motivate developers to contribute & further improve their work.
As a way forward, I welcome further discussion & ideas.
In reply to Chad Outten
Re: Modules and Plugins Database - an improved rating system?
by Matt Gibson -
Chad,
I've been thinking exactly the same thing! Gathering info from moodle registrations on popularity would be an ideal first step, as well as the ability to rate database entries with stars a bit like the forums. I think the current 0-100 scale is a bit arbitrary - what exactly does a 58 mean?
Also (slightly realted) the designation "1.6 or higher" is not much use if a plugin won't work with 1.9 yet. Votes/thoughts on MDLSITE-353 much appreciated
Matt
I've been thinking exactly the same thing! Gathering info from moodle registrations on popularity would be an ideal first step, as well as the ability to rate database entries with stars a bit like the forums. I think the current 0-100 scale is a bit arbitrary - what exactly does a 58 mean?
Also (slightly realted) the designation "1.6 or higher" is not much use if a plugin won't work with 1.9 yet. Votes/thoughts on MDLSITE-353 much appreciated
Matt
In reply to Matt Gibson
Re: Modules and Plugins Database - an improved rating system?
by Chad Outten -
@Matt: you got my vote!
i agree the '0-100' rating scale for plugins could better use the 'stars' system as per forums. i like your idea of collecting 'plugin popularity' when user registers their installation with moodle.org - this would definitely be a step in the right direction.
its useful data, but popularity doesn't necessarily reflect the stability of the plugin. similarly, its no doubt possible to collect data indicating the number of total, daily downloads (as per downloads page) for any given plugin. but again, the number of downloads doesn't always translate into quality.
both the above suggested methods would typically collect data from sysadmins & developers. does this bias data? surely educators & students are also entitled to rate plugins - from their user perspective. i appreciate this would add another layer of complexity to any rating system.
what do other ppl think?
i agree the '0-100' rating scale for plugins could better use the 'stars' system as per forums. i like your idea of collecting 'plugin popularity' when user registers their installation with moodle.org - this would definitely be a step in the right direction.
its useful data, but popularity doesn't necessarily reflect the stability of the plugin. similarly, its no doubt possible to collect data indicating the number of total, daily downloads (as per downloads page) for any given plugin. but again, the number of downloads doesn't always translate into quality.
both the above suggested methods would typically collect data from sysadmins & developers. does this bias data? surely educators & students are also entitled to rate plugins - from their user perspective. i appreciate this would add another layer of complexity to any rating system.
what do other ppl think?
In reply to Chad Outten
Re: Modules and Plugins Database - an improved rating system?
by Ger Tielemans -
I would prefer to start with a clearinghouse indication: what is the opnion of the Moodle HeadQuarters? Does an ad-onn folluw the rules for integration and future-proof?
In reply to Ger Tielemans
Re: Modules and Plugins Database - an improved rating system?
by Julian Ridden -
How do we define future proof? Thats a hard one.
A question I had in the past is do we allow only plugins under an open-source license or commercial apps as well.
Great comments on the changes to the rating system. So obvious when someone else mentions it
A question I had in the past is do we allow only plugins under an open-source license or commercial apps as well.
Great comments on the changes to the rating system. So obvious when someone else mentions it
In reply to Chad Outten
Re: Modules and Plugins Database - an improved rating system?
by Anthony Borrow -
Chad - Can you add a feature request or improvement issue to the tracker in the MDLSITE project and reference the discussion here. Thanks - Anthony
In reply to Anthony Borrow
Re: Modules and Plugins Database - an improved rating system?
by Chad Outten -
New tracker issue created @ MDLSITE-406
I'd encourage you to vote & watch the issue > if you believe we can improve the 'modules & plugins database' rating system.
Cheers, Chad
I'd encourage you to vote & watch the issue > if you believe we can improve the 'modules & plugins database' rating system.
Cheers, Chad
In reply to Chad Outten
Re: Modules and Plugins Database - an improved rating system?
by Martin Dougiamas -
Great!
As ideas surface, can there please be some brief analysis with them of
- how much is database template configuration and
- how much is new code in the database module
Cheers
As ideas surface, can there please be some brief analysis with them of
- how much is database template configuration and
- how much is new code in the database module
Cheers
In reply to Martin Dougiamas
Re: Modules and Plugins Database - an improved rating system?
by Chad Outten -
ok, here's my analysis...
Database template configuration
ratings: drop menu with values 0-100 exists
comments: text box for comments exists & is ok
suggested improvements:
New code in database module
Cheers,
Chad
Database template configuration
ratings: drop menu with values 0-100 exists
comments: text box for comments exists & is ok
suggested improvements:
- add field(s) for reviewers to favour & recommend extensions (radio buttons); values: yes, no
- add field for user type (drop menu); values: admin, developer, educator etc
- use a more meaningful rating scale eg. 1-7 stars as per moodle.org forums. is mapping of existing data possible?
- change 'comment' field title to 'review' & use subtext that will focus reviews eg. "having tested this extension, review it based on: ease of installation, functionality, stability, moodle-like appearance etc"
- criteria such as ease of installation, functionality, stability, moodle-like appearance & overall rating may be more relevant than 'usefulness'.
New code in database module
- count of downloads on a per extension basis will require new code
- calculations that display most favoured, recommended, reviewed extensions will require new code
Cheers,
Chad
In reply to Chad Outten
Re: Modules and Plugins Database - an improved rating system?
by Julian Ridden -
Great checklist Chad,
My fear here would now be that we are leaning towards over complication. I agree that a better assessment process would only be advantageous to the database, the degree to which you are describing seams to be overly complex.
Any chance of a mockup of what you are thinking?
My fear here would now be that we are leaning towards over complication. I agree that a better assessment process would only be advantageous to the database, the degree to which you are describing seams to be overly complex.
Any chance of a mockup of what you are thinking?
In reply to Julian Ridden
Re: Modules and Plugins Database - an improved rating system?
by Chad Outten -
ive done a quick mockup...3 steps
1. Developer adds extension to database
2. Potential users read description, discussion, check out demo &/or downloads, test for self
3. Users then review, rate, recommend, favour the extension
Ideally, data collected from the reviews could be calculated > so other potential users can see top rated, most reviewed, recommended, favoured extensions etc
1. Developer adds extension to database
2. Potential users read description, discussion, check out demo &/or downloads, test for self
3. Users then review, rate, recommend, favour the extension
Ideally, data collected from the reviews could be calculated > so other potential users can see top rated, most reviewed, recommended, favoured extensions etc
In reply to Chad Outten
Re: Modules and Plugins Database - an improved rating system?
by Julian Ridden -
Cool, that does look simple. I was afraid of over-complication but that looks great. I guess its not me your pitching at however, but it has my vote
In reply to Chad Outten
Re: Modules and Plugins Database - an improved rating system?
by Matt Gibson -
Looks good!
Can this be done with the existing database code? Presumably, extension and user would be auto filled, but what do user type and favour mean?
Can this be done with the existing database code? Presumably, extension and user would be auto filled, but what do user type and favour mean?
In reply to Chad Outten
Re: Modules and Plugins Database - an improved rating system?
by Anthony Borrow -
Chad - Again, thank you for pulling some of this together. I will also add some comments in the tracker but I wanted to summarize what I have garnered from this thread. There are various types of users (students, teachers, admins, developers, and possibly Moodle Partners. Some of the characteristics that we would like to see assessed are:
Creditibility - I think that credibility is a combination of some of the other factors such as stability, better maintained, and Moodle-like appearance so I would say remove that from the list and let folks determine for themselves what makes code credibile or a viable option worth considering.
Stability - In my mind refers to a general statement about the quality of the code. Does the code accomplish what it says that it does in a reliable manner. Does the code seem to be sufficiently mature (i.e. beyond alpha quality)? Stable code is code that is functional with a minimal amount of errors.
Better maintained - This can be tricky to assess. Does the project have a designated maintainer or team of maintainers who respond to issues in the tracker? It is difficult to simply look at the last change and determine the level of maintenance being done because some code does not require a great deal of maintenance. Similarly, some good code can have a number of issues in the tracker. So I do not think there is any easy way of determining how well code is being maintained. While there may be some obvious indicators when things are not being well maintained. For example, a high number of issues that have not been responded to in the tracker. Instead, I suspect that by and large this is a very subjective rating that is time sensitive. A project could have been maintained well and then the maintainer leaves. So I find myself wondering, recognizing the general importance of knowing this, how can we assess how well something is being maintained. Might the real question be: Does this project have an active maintainer? Would a yes or no be sufficient? By allowing users to rate this and with a date, we might see that a project is no longer being maintained. I would also add that if there is code in CONTRIB that is not being actively maintained the community is welcome to write to me or submit a request in the tracker that we seek a maintainer for the code.
Favor (favour) - Strikes me as an overall rating. Do I like it? Am I satisfied with it? I think this can be incorporated into the overall rating. If the overall rating is high then I assume the person favors it. If the rating is low then they do not. So I would recommend striking this one.
Popularity is difficult to assess. Two options were given. Base popularity on the number of downloads or on the number of times it shows up in registration. The problem with downloads is that not everything is hosted in CVS (although I am making a push this summer and encouraging folks to maintain as much code as possible using the CVS server). We can only count the number of downloads if the download.php file is used to help collect the statistics. The problem with trying to use Moodle registration is that not only would require some work to pull out the data from the registration records but it would also be difficult to get statistics on things like patches and other plugins which may not be modules. I think we would need to have a way for folks to indicate they areusing patches, plugins, etc. Perhaps simply a count of the number of ratings for a project could provide a general but limited sense of popularity.
Review/Comments - seems to be what is already available by allowing folks to comment on a module or plugin. The only thing I would do is to clarify that comments are not really to be questions about the module or plugin but rather statements of assessment. Questions and topics for discussion ought to go into the forums. Too often questions listed in the comments section simply do not get seen or noticed by the maintainer. So some clarity about what the comments are really for could be helpful.
Ease of Installation - In part this falls under documentation. I think that all of the modules and plugins should have a README file which includes basic installation instructions. In addition, I would like to see an entry in Moodle Docs. So for me the critical question has to do with documentation completion. Perhaps a scale of 1-10 would be helpful with 10 meaning fully documented and 0 meaning no documentation. Again, this is something that might change over time. So it would be possible to see that a project started out with poor documentation but eventually produced better or more complete documentation.
Functionality can mean a number of things from 'Does it work?', 'Is it useful (utility)?' to 'Does it do what it says it does?'. I think the question of it working in part is answered by an assessment of the quality of code. The question of utility strikes me as being related to favor or the overall rating. So I'm not sure that this is really needed.
Moodle-like appearance/feel - Essentially I think this is getting at the usability of project. Does it seem to fit with in Moodle and can folks use it pretty easily. I'm not convinced of the usefulness of this particular scale but perhaps a usability scale of 1 to 10 might be helpful. Perhaps some further discussion on this might be helpful.
Rules for integration - I think this has to do with a combination of quality of code and usability but I'm not clear as to what this is really assesses. Is it tied in with the concept of future proof?
Future-proof - I'm not sure what this is. I am guessing it has to do with how actively the code is being maintained and if there is a commitment to ensure that it will work for the version of Moodle. I would argue that it is covered by having an active maintainer but perhaps Ger (or others) have something else in mind.
I hope that if folks follow the Guidelines for Contributed Code then we can at least partially help to ensure that code is functionally stable (since I try to check and make sure that code does what it says it does beore I add it to CONTRIB), well maintained (by using the tracker), and somewhat documented (by using Moodle docs). Let's continue to discuss and clarify our ideas and then work toward some type of implementation. Peace - Anthony
- credibility
- stability (quality of code: 1-10)
- better maintained (active maintainer: Y/N)
- favor
- overall rating (1-10)
- popularity either based on number of downloads or number of registered sites (count ratings)
- review/comments
- ease of installation (documentation: 1-10)
- functionality
- Moodle-like appearance/feel (usabiility: 1-10)
- roles for integration (quality of code+usability?)
- future proof (active maintainer?)
Creditibility - I think that credibility is a combination of some of the other factors such as stability, better maintained, and Moodle-like appearance so I would say remove that from the list and let folks determine for themselves what makes code credibile or a viable option worth considering.
Stability - In my mind refers to a general statement about the quality of the code. Does the code accomplish what it says that it does in a reliable manner. Does the code seem to be sufficiently mature (i.e. beyond alpha quality)? Stable code is code that is functional with a minimal amount of errors.
Better maintained - This can be tricky to assess. Does the project have a designated maintainer or team of maintainers who respond to issues in the tracker? It is difficult to simply look at the last change and determine the level of maintenance being done because some code does not require a great deal of maintenance. Similarly, some good code can have a number of issues in the tracker. So I do not think there is any easy way of determining how well code is being maintained. While there may be some obvious indicators when things are not being well maintained. For example, a high number of issues that have not been responded to in the tracker. Instead, I suspect that by and large this is a very subjective rating that is time sensitive. A project could have been maintained well and then the maintainer leaves. So I find myself wondering, recognizing the general importance of knowing this, how can we assess how well something is being maintained. Might the real question be: Does this project have an active maintainer? Would a yes or no be sufficient? By allowing users to rate this and with a date, we might see that a project is no longer being maintained. I would also add that if there is code in CONTRIB that is not being actively maintained the community is welcome to write to me or submit a request in the tracker that we seek a maintainer for the code.
Favor (favour) - Strikes me as an overall rating. Do I like it? Am I satisfied with it? I think this can be incorporated into the overall rating. If the overall rating is high then I assume the person favors it. If the rating is low then they do not. So I would recommend striking this one.
Popularity is difficult to assess. Two options were given. Base popularity on the number of downloads or on the number of times it shows up in registration. The problem with downloads is that not everything is hosted in CVS (although I am making a push this summer and encouraging folks to maintain as much code as possible using the CVS server). We can only count the number of downloads if the download.php file is used to help collect the statistics. The problem with trying to use Moodle registration is that not only would require some work to pull out the data from the registration records but it would also be difficult to get statistics on things like patches and other plugins which may not be modules. I think we would need to have a way for folks to indicate they areusing patches, plugins, etc. Perhaps simply a count of the number of ratings for a project could provide a general but limited sense of popularity.
Review/Comments - seems to be what is already available by allowing folks to comment on a module or plugin. The only thing I would do is to clarify that comments are not really to be questions about the module or plugin but rather statements of assessment. Questions and topics for discussion ought to go into the forums. Too often questions listed in the comments section simply do not get seen or noticed by the maintainer. So some clarity about what the comments are really for could be helpful.
Ease of Installation - In part this falls under documentation. I think that all of the modules and plugins should have a README file which includes basic installation instructions. In addition, I would like to see an entry in Moodle Docs. So for me the critical question has to do with documentation completion. Perhaps a scale of 1-10 would be helpful with 10 meaning fully documented and 0 meaning no documentation. Again, this is something that might change over time. So it would be possible to see that a project started out with poor documentation but eventually produced better or more complete documentation.
Functionality can mean a number of things from 'Does it work?', 'Is it useful (utility)?' to 'Does it do what it says it does?'. I think the question of it working in part is answered by an assessment of the quality of code. The question of utility strikes me as being related to favor or the overall rating. So I'm not sure that this is really needed.
Moodle-like appearance/feel - Essentially I think this is getting at the usability of project. Does it seem to fit with in Moodle and can folks use it pretty easily. I'm not convinced of the usefulness of this particular scale but perhaps a usability scale of 1 to 10 might be helpful. Perhaps some further discussion on this might be helpful.
Rules for integration - I think this has to do with a combination of quality of code and usability but I'm not clear as to what this is really assesses. Is it tied in with the concept of future proof?
Future-proof - I'm not sure what this is. I am guessing it has to do with how actively the code is being maintained and if there is a commitment to ensure that it will work for the version of Moodle. I would argue that it is covered by having an active maintainer but perhaps Ger (or others) have something else in mind.
I hope that if folks follow the Guidelines for Contributed Code then we can at least partially help to ensure that code is functionally stable (since I try to check and make sure that code does what it says it does beore I add it to CONTRIB), well maintained (by using the tracker), and somewhat documented (by using Moodle docs). Let's continue to discuss and clarify our ideas and then work toward some type of implementation. Peace - Anthony
Thanks everyone for your great comments and suggestions. 
As mentioned in MDLSITE-406, rather than spending time customizing the existing modules and plugins database activity, the best course of action seems to be to look for an alternative solution, in order to to provide additional functionality. Please see MDLSITE-571. All comments and suggestions are welcome!
As mentioned in MDLSITE-406, rather than spending time customizing the existing modules and plugins database activity, the best course of action seems to be to look for an alternative solution, in order to to provide additional functionality. Please see MDLSITE-571. All comments and suggestions are welcome!
I'd suggest something similar to that which the vBulletin forum software uses. Not sure how much you can see without having to register but it's at :
The forums are set up for different versions, themes and plug-ins. The allow you to tag ones you're interested in and also keep track of which ones you've downloaded to use (and they prompt you to mark them that they've been installed). As well as all this there's a rating system plus how many downloads etc.
It's really useful and easy to use/navigate. If it could be adapted for Moodle use that would be great !
In reply to Chad Outten
Re: Modules and Plugins Database - an improved rating system?
by Robert Brenstein -
Yes, excellent idea. I like the multi-dimensionality, I mean having multiple assessment and categorization criteria. This would be a nifty feature for forums as well.
In reply to Chad Outten
Re: Modules and Plugins Database - an improved rating system?
by Darren Tams -
I think a rating system is essential.
Having just setup a Joomla server, I was disapointed with the downloads section of moodle, which seams happy just to list plugins.
I think the joomla system is what you should be aiming for and allow good plugins to flurish and can only help Moodle.
I like Moodle, I think its excellent in what it does and look forward to the future.
Having just setup a Joomla server, I was disapointed with the downloads section of moodle, which seams happy just to list plugins.
I think the joomla system is what you should be aiming for and allow good plugins to flurish and can only help Moodle.
I like Moodle, I think its excellent in what it does and look forward to the future.
It would be neat if there was a way to sort the plugins by compatability. Why use up time looking at plugins that aren't even compatable with 1.9.4?
The Moodle community is great. Thanks to all the plugin developers for your contrubutions.