Particularly helpful Moodler calculation amendments

Particularly helpful Moodler calculation amendments

by Helen Foster -
Number of replies: 48
Picture of Core developers Picture of Documentation writers Picture of Moodle HQ Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers Picture of Translators

You may be aware that particularly helpful Moodler group membership is based on useful ratings of forum posts and is calculated automatically. A lot more posts are being rated as useful these days and so we have amended the calculation a little to hopefully improve the results. In particular, only ratings for replies (not first posts in discussions) are counted now, and a user must have posted something rated as useful in the past 6 months.

Your feedback on the current Particularly helpful Moodlers group membership is welcome, especially if you notice anything not quite right. Please let us know so we can review things and, if necessary, make further amendments to the calculation.

Average of ratings: Useful (1)
In reply to Helen Foster

Re: Particularly helpful Moodler calculation amendments

by Séverin Terrier -
Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Testers Picture of Translators

There's one thing i don't really understand : why don't you count the first post of a discussion?

When someone has encountered a problem, tested a solution, and make a (new) discussion to let everyone know about this useful information, he isn't rewarded for that sad

Average of ratings: Useful (7)
In reply to Séverin Terrier

Re: Particularly helpful Moodler calculation amendments

by Nicolas Martignoni -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers Picture of Translators

I agree with Séverin.

Another frequent use case: someone sponteanously reporting a best practice or tips and tricks in a new discussion.

The other modifications to the calculation of PHM are very sensible though. Was there some evaluation to know the impact of not taking first posts in account? I'm not (yet?) convinced that these make a difference.

Average of ratings: Useful (2)
In reply to Nicolas Martignoni

Re: Particularly helpful Moodler calculation amendments

by john Simpson -

To recollect for me giving usefuls in first posts.

Our quiz expert demontrating improvement on our moodle quiz plugins

Someone offering  a free book to the moodle community on how to use moodle

Someone showing a link of their well presented good looking moodle site, confirming against all competitors that moodle doesn't have to be ugly.

Any good tips are always appreciated by readers.

I always click useful if I think the post is useful. Hopefully users are not clicking useful simply due to agreeing with someones opinion for example. Useful means useful, that's it.

Average of ratings: Useful (2)
In reply to john Simpson

Re: Particularly helpful Moodler calculation amendments

by Itamar Tzadok -

Some may find it useful to see that others are of the same opinion. Some (myself included) mark useful posts that make them laugh. You actually put it in the best tautological way: useful means useful, that's it. It is just that this "it" can be many many things and very likely not the same thing for everyone. smile

In reply to Nicolas Martignoni

Re: Particularly helpful Moodler calculation amendments

by Ray Morris -

As Séverin and Nicolas have said, I'm having trouble imagining why you'd not count a post that's been marked "useful" if it happens to be the first post in the a thread.  I must be missing something.  It seems that pushes the forums more toward being strictly a question-and-answer support resource, rather than a discussion community.  You don't want to recognize people posting useful things like Moodle news, interesting new ways of doing things in Moodle, warnings about gotchas, etc?


john Simpson said this thread is a good example:

https://moodle.org/mod/forum/discuss.php?d=278302


Average of ratings: Useful (3)
In reply to Séverin Terrier

Re: Particularly helpful Moodler calculation amendments

by Itamar Tzadok -

This becomes even more interesting if that someone forgets the solution, encounters a similar problem, goes back to the forum and finds his/her post and thus solves the problem at hand, but cannot rate his/her own post as useful. smile

Average of ratings: Useful (1)
In reply to Itamar Tzadok

Re: Particularly helpful Moodler calculation amendments

by john Simpson -

If set, he or she will find the answer post in their email.

In reply to john Simpson

Re: Particularly helpful Moodler calculation amendments

by Itamar Tzadok -

The use case is such that he/she is the OP. OPs typically don't receive an email with their post. And even if they could receive such email, I for one would not opt for it. We have enough emails as it is. smile

In reply to Séverin Terrier

Re: Particularly helpful Moodler calculation amendments

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

I agree with everyone else that at first site, there is no logical reason for not counting the first post in a thread.

However, it is not a matter of logic. It is a matter of data. I discussed this with Helen and Dan P, and apparently when you look at the actual data from moodle.org, the changes they made give you better results when you look at who ends up in the PHM group. Can't argue with that.

In reply to Tim Hunt

Re: Particularly helpful Moodler calculation amendments

by Nicolas Martignoni -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers Picture of Translators

> (...) the changes they made give you better results when you look at who ends up in the PHM group.

In understand this, but you can't justify an action just because it allows to hit the target. In french we would say: "la fin ne justifie pas les moyens".

Three changes have been made in this calculation:

  • one should have posted something useful (instead of simply posting a "unuseful" one): this should effectively reduce the number of PHMs,
  • the time frame for posting such a message has been extended to 6 months instead of 60 day, which would tend to increase the number of PHMs,
  • the first posts aren't taken in account in the calculation. This one seems too me rather illogical.

What I'm asking is what is the result if you don't make the third one? And, why not amend the second one, e.g. instead of 6 months, extend the time frame for posting useful messages to 120 days (or any other duration) only?

Were these possibilities tested?

Average of ratings: Useful (1)
In reply to Nicolas Martignoni

Re: Particularly helpful Moodler calculation amendments

by Séverin Terrier -
Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Testers Picture of Translators

I totally agree with what Nicolas says, and asks.

I don't find logical to extend from 60 days to 6 months : this doesn't encourage people to post regularly. But it allows people making just very few messages, but (nearly) always useful to be taken in account...

And what would change if first posts are taken in the calculation?

In reply to Séverin Terrier

Re: Particularly helpful Moodler calculation amendments

by Helen Foster -
Picture of Core developers Picture of Documentation writers Picture of Moodle HQ Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers Picture of Translators

Thanks everyone for your comments and feedback.

Séverin, you asked why we don't count first posts in discussions. The reason why we excluded them was because we've noticed a lot of first posts rated as useful which don't make sense i.e. it seems they were done randomly or by someone who didn't understand the system.

First posts can of course still be rated as useful, giving the poster feedback. Also, as illustrated in this thread, there can be many reasons for rating a post as useful.

Nicolas, to explain a bit more about the reasons for the changes, the requirement to have posted something rated as useful was because we noticed people being made particularly helpful Moodler and being awarded a PHM 2014 badge when they hadn't actually made a single useful post in the whole of 2014. We tried setting a requirement to have posted something rated as useful in 60 days, and in 120 days, but both of these requirements resulted in a lot of people no longer being PHMs. We removed the requirement to have made one post in 60 days, as people are unenrolled from a course after a period of 60 days inactivity and are then automatically removed from the PHM group anyway. We also tried increasing the number of rated posts and the ratio of total ratings to total posts. The figures used in the calculation were chosen arbitrarily previously and, as in the past, with the large amount of data involved, all we could do was make a change and observe the results.

As I mentioned in my initial post, if anyone notices anything not quite right in the Particularly helpful Moodlers group membership, we're happy to make further amendments to the calculation. Feel free to contact me via messaging if you prefer.

Average of ratings: Useful (2)
In reply to Helen Foster

Re: Particularly helpful Moodler calculation amendments

by dawn alderson -

Helen, hi

First, I think everyone here appreciates the work put into this, and that appreciation is extended to TH too.

I would like to offer an outsider viewpoint....given I am relatively new here.

When I used to award kids a gold star....over and over again...they ended up sticking them on their faces!

And the student who gets a consistent C.....would declare, quite often, an affinity/attraction for the pub rather than the library.

I received a helpful moodler badge...and I have no idea to this day why?  I get mod-ed more than I get usefuls wink

The point is, if the recipient has no idea about the what, why, how, when etc in terms of reward......but only knows the punishment in a system...then they will play to their strengths.....really it is true......maths/algorithms.....system protocol......can make peeps yawn unless it is attached directly to their experiences where they can easily recognise the value of their input, or at least find value in reflective activity from an experience.

The maths/algorithms are sound...the ethics needs more work.

hope helpful

Dawn   

In reply to Helen Foster

Re: Particularly helpful Moodler calculation amendments

by Nicolas Martignoni -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers Picture of Translators

Thanks Helen for these very useful (!) explanations. I understand now the reason of the first post thing (though I find it strange that so many people mis-evaluate first posts as useful smile).


In reply to Tim Hunt

Re: Particularly helpful Moodler calculation amendments

by Ray Morris -

However, it is not a matter of logic. ...

> the changes they made give you better results when you look at who ends up in the PHM group. Can't argue with that.


I understand what you're saying.  Just for fun, I'd like to try arguing with that:


class forum_post {
    function get_author_name() {
        return 'Ray Morris';
    }
    function get_author_country() {
        if(substr($this->get_author_name(), 1) == 'R') {
        return 'US';
    }
    }
}


I tested that with against an earlier post in this thread and it gives reasonably accurate results at the moment. That must be the right way to do it.  Can't argue with that. big grin


The functions above are silly examples, of course, but I think they illustrate a valid point. The scoring algorithm might work okay (or not), but the argument that it happens to give reasonable-looking results at the moment isn't al that persuasive, to me.





Average of ratings: Useful (3)
In reply to Ray Morris

Re: Particularly helpful Moodler calculation amendments

by dawn alderson -

Ray, yes.

But it helps to elaborate when posting code in terms of meaning, otherwise it is a different language for many here, always nice to share meaning.

I get that meaning.  Thinking about differentiation:

-helpful

-informed

-funny  

Across the board.

I understand such diff is embedded practice in the US....across online forums.....and on.

And on a funny note......being inspired by the US model here....I just love this clip (from 1.11 in...sorry...off topic tongueout




Average of ratings: Useful (1)
In reply to Séverin Terrier

Re: Particularly helpful Moodler calculation amendments

by Visvanath Ratnaweera -
Picture of Particularly helpful Moodlers Picture of Translators
This is pretty useful.

Séverin, with the five bullets you've earned already, your seat in Olymph is gurenteed!
In reply to Visvanath Ratnaweera

Re: Particularly helpful Moodler calculation amendments

by Séverin Terrier -
Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Testers Picture of Translators

That's where i see things illogical/ironical : my post is not the first one (so votes are counted), it has several "useful" ranting, but these means more "+1, i agree with that"... so shoulnd't really be counted.

We can't coun't exactly which information IS really useful, but just try to estimate things, based on calculations and estimations...

In reply to Helen Foster

Re: Particularly helpful Moodler calculation amendments

by Visvanath Ratnaweera -
Picture of Particularly helpful Moodlers Picture of Translators
Helen

Just to avoid a misunderstanding: You are not talking about hacking the source of moodle.org to disable the "Rate as useful" drop-down for posts which start a discussion, rather about an amendment to the algorithm of making one a Particularly helpful Moodler, right?

Before going in to details, how is it possible to make a difference beween useful in moodle.org but not counted as a qualification for a PHM? If an OP could be useful, then by posting a useful OP somebody makes himself useful. If an OP can not be useful, then why is there a "Rate as Useful" button in the first place? Apart from the many reasons other have braught, I have two arguments for the usefulness of an OP:
1. An OP need not to be a problem or a question, it could very well be a report, a summary, a write-up solving a problem.
2. A good question is a good thing, as all teachers know (but not allways willing to accept ;).

Now to the "algorithm": One useful rating every 6 months. I don't find it an exciting one, something easily settled with your office-mate, for example, for a cup of coffee every 6 months.
smile

No, seriously. First thing, what is at stake? I have this "badge" for many years and can not think of a single reason why it (the badge) is useful. At first glance one might think of a (military) breast plate. The casual visitor (the commoner) knows that this is an important person. But if one clicks on the badge, he's told, "These people have written a lot of useful posts in the forums recently, as determined by the Moodle community...". Then, if the visitor is that observant, he could get the whole list forum posts by any person. Or, is the badge for the PHM himself, kind of a reminder to be seious? Oops, that shouldn't limit my freesom to crack a joke!

Whatever the use of the badge, it has been there a very long time. As I understand, the current initiative is to _automate_ it. Now that is highly explosive: moodle.org forums are well-known to be polite and helpful. That doesn't come from nothing. That is a culture which was set and nursed by an equally cultivated team of leaders. A personal touch. Now our Community Leadress comes up with this bright idea of lending that to a bot. Sorry, I am not the least inclined to talking to bots, please releave take me out of that group before that happens. Just a passing quesion, aren't you making yourself superflous by automating your job?

P.S. Yet another painful discussion for discussions' sake in the (amalgamated) Moodle community sites form. Now I've spoken, I can forget the topic and spend that time for something Useful (not a typo ;()
In reply to Visvanath Ratnaweera

Re: Particularly helpful Moodler calculation amendments

by Nicolas Martignoni -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers Picture of Translators

Hi Visvanath,

I respect you point of view about all what you said, even if I don't share it. But this post is about the calculation of the PHMs. May I respectfully suggest you start a new discussion to address your points?

PS. Your evaluation of my (and others) messages ("painful") in this discussion is not useful at all.

In reply to Nicolas Martignoni

Re: Particularly helpful Moodler calculation amendments

by dawn alderson -

Dogs are useful too.  They can retrieve the mail and papers in the morning...very useful.

Average of ratings: Useful (2)
In reply to dawn alderson

Re: Particularly helpful Moodler calculation amendments

by Itamar Tzadok -

Add that to the calculation ... big grin

In reply to Helen Foster

Re: Particularly helpful Moodler calculation amendments

by Itamar Tzadok -

Maybe it's time to add more adjectives to the post rating:

  • Entertaining - posts that make you laugh (group: Particularly Entertaining Moodler (PEM)).
  • Artistic - posts which please you aesthetically (group: Particularly Artistic Moodler (PAM)).

Such ratings could allow a finer appreciation of the value of the post. The groups and badges for these ratings are nice to have but not necessary.

smile


In reply to Helen Foster

Re: Particularly helpful Moodler - PHM 2015 badge

by Séverin Terrier -
Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Testers Picture of Translators

Hi,

I just have one related question, to PHM and the "PHM 2015 badge". How and when this badge is delivered?

This question because i'm in the PHM group, but don't have the badge. And it seems people didn't receive the badge at the same time...

Séverin

In reply to Séverin Terrier

Re: Particularly helpful Moodler - PHM 2015 badge

by Helen Foster -
Picture of Core developers Picture of Documentation writers Picture of Moodle HQ Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers Picture of Translators

Hi Séverin,

We've not yet got to the stage of being able to award the PHM 2015 badge automatically - it has to be done manually, and is quite time-consuming. Thus, please be patient; you will receive your badge, definitely before the end of 2015. wink

In reply to Helen Foster

Re: Particularly helpful Moodler - PHM 2015 badge

by Séverin Terrier -
Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Testers Picture of Translators

Hi Helen,

Ok, i better understand then. When automated, it will save your time.

I'm patient, but just like to understand things smile

Thanks for the badge wink

In reply to Helen Foster

Re: Particularly helpful Moodler - PHM 2015 badge

by Visvanath Ratnaweera -
Picture of Particularly helpful Moodlers Picture of Translators
Hi Helen

Does that mean I might get a badge even if I don't want one?
In reply to Visvanath Ratnaweera

Re: Particularly helpful Moodler - PHM 2015 badge

by Przemek Kaszubski -
Picture of Particularly helpful Moodlers Picture of Testers

There must be a way to take first posts into account.

I think I did deserve my two usefuls from this post, for example :

https://moodle.org/mod/forum/discuss.php?d=279544

What should we do if we have an idea to share..? Ask if anybody's interested before we post it? Search for somebody else's discussion and attach it there? Unuseful options, IMO.

Przemek

Average of ratings: Useful (1)
In reply to Przemek Kaszubski

Re: Particularly helpful Moodler - PHM 2015 badge

by john Simpson -

What I normally do is to click useful on the thread writers 2nd post, for his first post if it's useful.

I get annoyed with someone who starts a thread and never manages its by answering his/her response of posts, so perhaps it is a good idea not to include first posts.

And thats what I've done with your thread, by clicking on your second post.

In reply to Przemek Kaszubski

Re: Particularly helpful Moodler - PHM 2015 badge

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

I don't think you really need to worry.

My guess would be that anyone who gets a PHM badges does not just creep over the threshold that has been set. I expect most people who have the badge have smashed thought the minimum requirements, and a long way beyond.

So, the details do not really matter.

Average of ratings: Useful (1)
In reply to Tim Hunt

Re: Particularly helpful Moodler - PHM 2015 badge

by Visvanath Ratnaweera -
Picture of Particularly helpful Moodlers Picture of Translators
All

Good guess by Tim:
> My guess would be that anyone who gets a PHM badges does not just creep over the threshold that has been set. I expect most people who have the badge have smashed thought the minimum requirements, and a long way beyond.

My first guess is that they are way beyond the minimum requirements because they are first helpers, badege came later.

My second guess is that the new automated badge creation will invite a whole generation of badge collectors.

> So, the details do not really matter.

I'm sorry, it is the opposite. I am telling it a third time: Helen's automation will back fire!
In reply to Visvanath Ratnaweera

Re: Particularly helpful Moodler - PHM 2015 badge

by Przemek Kaszubski -
Picture of Particularly helpful Moodlers Picture of Testers

If I ask a question in the first post of a new discussion ...

and then reply to myself with a solution ...

Would that count ? smile

I'm joking here. I don't, and didn't, mean to distract anyone.

I do find this long discussion useful smile also because I can't really decide whether or not to bother with badges in my (blended learning!) courses or not, whether to automate the procedure or not, etc etc.

I also once asked this question - whether it is possible for the grader to retrieve all those posts rated by him/her as Useful:  https://moodle.org/mod/forum/discuss.php?d=279578.  

Kind regards to All.

Average of ratings: Useful (2)
In reply to Przemek Kaszubski

Re: Particularly helpful Moodler - PHM 2015 badge

by john Simpson -

For students in blended learning I do believe it to be a good idea to award badges in forums.It hopefully  gives encouragement for students to participate with good posts.

I know you were only joking, but if you came up with a very good question, and by chance gave the perfect answer, yes I'm certain you will get some clicks. It's all fun, but why we all respond here on this forum has nothing to with the badges. If the badges disappeared tomorrow, we'd all still be here, doing what we do.

Average of ratings: Useful (2)
In reply to john Simpson

Re: Particularly helpful Moodler - PHM 2015 badge

by Marcus Green -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers

"If the badges disappeared tomorrow, we'd all still be here, doing what we do." That is certainly  true for me, but  I still like having badges and wonder what you need to do to get others. When I first noticed this thread I assumed that PHM was Pointy Haired Manager which is not a badge type I would associate with these forums smile

In reply to Marcus Green

Re: Particularly helpful Moodler - PHM 2015 badge

by Przemek Kaszubski -
Picture of Particularly helpful Moodlers Picture of Testers

I think, the PHM badge can add to the fun and motivation of late joiners here, like myself. But I primarly like to observe which posts or info tend to be marked as useful - it's a form of learning about Moodlers, as well Moodle..

In reply to Przemek Kaszubski

Re: Particularly helpful Moodler - PHM 2015 badge

by Przemek Kaszubski -
Picture of Particularly helpful Moodlers Picture of Testers

I've been awarded my PHM badge today - Yay!

In reply to Przemek Kaszubski

Re: Particularly helpful Moodler - PHM 2015 badge

by ben reynolds -

Congratulations! And, it retroactively appears on all your posts.

In reply to Przemek Kaszubski

Re: Particularly helpful Moodler - PHM 2015 badge

by Visvanath Ratnaweera -
Picture of Particularly helpful Moodlers Picture of Translators
Congrats!

And possibly one of the last badges offered by the fairies personally. The process is to be automated soon!
In reply to john Simpson

Re: Particularly helpful Moodler - PHM 2015 badge

by Visvanath Ratnaweera -
Picture of Particularly helpful Moodlers Picture of Translators
John

Yours must be the first time an answer to a joke collected the same number of points as the joke iteself!
In reply to Przemek Kaszubski

Re: Particularly helpful Moodler - PHM 2015 badge

by Visvanath Ratnaweera -
Picture of Particularly helpful Moodlers Picture of Translators
Przemek,

It is a good joke for sure. It has brought two "Useful" ratings already.
smile
In reply to Tim Hunt

Re: Particularly helpful Moodler - PHM 2015 badge

by Chris Collman -
Picture of Documentation writers

Well, I am not worried. I have sort of retired from Moodle.org but still get threads.

On the other end of the scale, maybe Helen needs to create a Helpful Moodler Emeritis badge?  There are a few people I can think of who deserve this way more than me smile

Chris

In reply to Visvanath Ratnaweera

Re: Particularly helpful Moodler - PHM 2015 badge

by Derek Chirnside -

I believe so Visvanath.  You can get a badge even if you don't want one.

In reply to Derek Chirnside

Re: Particularly helpful Moodler - PHM 2015 badge

by Visvanath Ratnaweera -
Picture of Particularly helpful Moodlers Picture of Translators
Derek

Is it heavy?
;-P
In reply to Visvanath Ratnaweera

Re: Particularly helpful Moodler - PHM 2015 badge

by Przemek Kaszubski -
Picture of Particularly helpful Moodlers Picture of Testers

Ben and Visvanath - thanks !

In reply to Przemek Kaszubski

Re: Particularly helpful Moodler - PHM 2015 badge

by ben reynolds -

But, of course, I will not mark your thanks "Useful" cool

In reply to ben reynolds

Re: Particularly helpful Moodler - PHM 2015 badge

by Derek Chirnside -

You guys are crazy.  With far too much spare time or else you type far to fast for your own good.

I will tell you a little story however.

I was a little interested when badges first came out on Moodle.  There was no rationale for them.  Just a decision out of the blue as far as the public record is concerned that 'we shall have them', and I presume someone else paid for them (ie I hope not from the 10% given to HQ).    I try to keep clutter down in my life (failing generally) and so I've never opted in for push notifications for any smart phone apps for badges and the like until three weeks ago.

FitBit zip.  https://www.fitbit.com/nz/zip/specs  My effort to fulfil a new year resolution.

A wireless activity tracker.  Apart from the fact that my smart phone is too old to do everything properly (a fact that was not even in the small print) it has been an interesting expedience.  A competition between my sloth and my desire to get fitter.

The I started getting badges.

Gumshoe (my first few hundreds), a success for my first goals, then an "Over achiever" badge when I dramatically exceeded my goal, and some emails and texts as well.  Fascinating, I actually liked getting these.  I can actually put myself in to the position of a learner, and it may be actually quite good.  Sometimes.  I presume someone may point to some research that it is good and worth the effort.

I'm pleased to see Przemek's badge arrive at last.  I liked a lot of his posts since he did deserve it, but it still took a while for it to arrive.

-Derek



Average of ratings: Useful (1)
In reply to Derek Chirnside

Re: Particularly helpful Moodler - PHM 2015 badge

by Przemek Kaszubski -
Picture of Particularly helpful Moodlers Picture of Testers

Who's crazy here, Derek? ;) (I just had a look at that website)

Thanks so much for your support and votes - I don't think I waited too long smile

Must say one thing - I now need to think twice before posting any tips or advice. The badge has its secret price attached smile 

Przemek

In reply to Przemek Kaszubski

Re: Particularly helpful Moodler - PHM 2015 badge

by Visvanath Ratnaweera -
Picture of Particularly helpful Moodlers Picture of Translators
Yes, yes, this subthread is about the weight of the badge. (FitBit zip, Derek's new year resolution, was about another weight, I'm afraid. ;)