Hello world,
I'd like to announce here that I am going to try and rewrite the great Workshop module for Moodle 2.0. Yes, I know about the other projects at the same area. Of course, I am open for cooperation.
My goal is to transform the Workshop into an easy-to-setup and intuitive activity while keeping almost all its mighty features and flexibility.
I have just started to work on the specification of functionality, implementation, database structure, user interface etc.
Any comments, ideas and improvements requests from the community are warmly welcome. If you have experiences with the module, please share! What did work for you and what did not? Are there some settings you have never used and could possibly be removed? Did you miss some feature?
The specification will be available at http://docs.moodle.org/en/Development:Workshop_module
Yet another attempt to rewrite Workshop for 2.0
Number of replies: 11
Re: Yet another attempt to rewrite Workshop for 2.0
Best wishes with this.
Is it feasible or desirable to make this backwards compatible? Could the existing module be relegated to Contrib as a plug in for those who need it. Or, remain as is i.e. hidden on new installs for a while before eventual removal?
The new module could then proceed unencumbered by the legacy issues. (As a Peer Review assignment type?)
Just had a quick look at the mind map. I don't understand the comment sin the screen shot. One of the reasons that the current module has limited utility is that the grade calculations are now hidden from users.
Is it feasible or desirable to make this backwards compatible? Could the existing module be relegated to Contrib as a plug in for those who need it. Or, remain as is i.e. hidden on new installs for a while before eventual removal?
The new module could then proceed unencumbered by the legacy issues. (As a Peer Review assignment type?)
Just had a quick look at the mind map. I don't understand the comment sin the screen shot. One of the reasons that the current module has limited utility is that the grade calculations are now hidden from users.
Re: Yet another attempt to rewrite Workshop for 2.0
Hi Ray,
at this stage, it is planned to have the new Workshop 2.0 being backwards compatible as much as possible. Please note, the existing module version has already been moved to contrib and the goal of my project is to get the source code into such a quality that it is taken back into the core
I am not afraid too much (yet
) of legacy issues. IMHO the Workshop has a great educational potential and when the UI is improved (made more usable, intuitive), it will not be more difficult to use than the other modules.
Regarding the attached comment: the branch is about "bias" and "reliability" settings as described in Using Moodle, chapter 6. The sentence says that that these settings actually make the calculation masked. So the goal is to rethink their meaning and value. AFAIK these two settings were already modified (merged?) since 1.6. And yes - I totally agree with you that the module must explain its calculation.
at this stage, it is planned to have the new Workshop 2.0 being backwards compatible as much as possible. Please note, the existing module version has already been moved to contrib and the goal of my project is to get the source code into such a quality that it is taken back into the core
I am not afraid too much (yet
) of legacy issues. IMHO the Workshop has a great educational potential and when the UI is improved (made more usable, intuitive), it will not be more difficult to use than the other modules.Regarding the attached comment: the branch is about "bias" and "reliability" settings as described in Using Moodle, chapter 6. The sentence says that that these settings actually make the calculation masked. So the goal is to rethink their meaning and value. AFAIK these two settings were already modified (merged?) since 1.6. And yes - I totally agree with you that the module must explain its calculation.
Re: Yet another attempt to rewrite Workshop for 2.0
Hi David,
great that you start working at the workshop module.
There is one issue I miss in the mindmap: selfgrading the own work by students.
This feature is very interesting for pedagogical concepts.
great that you start working at the workshop module.
There is one issue I miss in the mindmap: selfgrading the own work by students.
This feature is very interesting for pedagogical concepts.
Re: Yet another attempt to rewrite Workshop for 2.0
Hi Ralf
this is what "Students can perform self-assessment" feature (the 5th feature branch) should mean. Yes, self-evaluation is very strong tool and it will be retained.
Thanks for the comment!
this is what "Students can perform self-assessment" feature (the 5th feature branch) should mean. Yes, self-evaluation is very strong tool and it will be retained.
Thanks for the comment!
Re: Yet another attempt to rewrite Workshop for 2.0
Hey David,
Let me know if and when you need some testing resources, I'd be very keen to see this rewritten and would be happy to provide some functional/bug searching services. I'll check the mindmap at home tonight and see if there's anything I can add
Mark.
Let me know if and when you need some testing resources, I'd be very keen to see this rewritten and would be happy to provide some functional/bug searching services. I'll check the mindmap at home tonight and see if there's anything I can add
Mark.
Re: Yet another attempt to rewrite Workshop for 2.0
Thanks in advance Mark!
Re: Yet another attempt to rewrite Workshop for 2.0
Dear David,
thank you for taking on this task. I have always found the workshop quite interesting and there is still no other CMS that has a comparable feature. But to be quite frank, I gave up using it after two or three tries because I found it too complicated. This was quite a while ago, so I can only tell you in an abstract way what I found so hard back then.
Sorry for this lack of precision in my comments but I guess I can put this all in a nutshell by saying: I would put much more work into the user interface than into functionality. Most of the teachers I know are fascinated by the workshop module but quickly give up once they see all the options and the way that settings and results are displayed.
Good luck for your programming efforts, if you need a look at a prototype by an interested user, let me know. By the way, where do I find the mind map that Ray is talking about?
Best regards
Peter
thank you for taking on this task. I have always found the workshop quite interesting and there is still no other CMS that has a comparable feature. But to be quite frank, I gave up using it after two or three tries because I found it too complicated. This was quite a while ago, so I can only tell you in an abstract way what I found so hard back then.
- First, I remember that I found it hard to tell the different phases of a workshop apart and set an appropriate time frame. Maybe a first step would be to not force teachers to set all the due dates in the first menu. I would tend to just write the basic objectives an some meta data on the 2st page (Title, description etc.) and then think about all the due dates as I am setting up the exercise and the criteria.
- Secondly, as others have mentioned, the grading was not really transparent to me. I am pretty sure an easy-to-use graphical representation of the way a grade is calculated on the basis of the current setting would be great (for example a pie chart that changes as soon as you change settings?).
- The third problem may be something that just may not be something that can be solved or that maybe even I did not get, so bear with me if I am writing nonsense now: I got the impression that the workshop module needs a sort of critical mass of students in order to function properly. If you have really small classes of, say, 6 students, the workshop module does not have all the functions I would need. One example: In a small class, for example, it makes more sense to me if the teacher says who assesses what rather than refer this task to an algorithm. In a way, this would be a simplification because the teacher makes simple decisions just like he would in a groupwork situation in a classroom. But again, maybe the module does things which I did not get.
Sorry for this lack of precision in my comments but I guess I can put this all in a nutshell by saying: I would put much more work into the user interface than into functionality. Most of the teachers I know are fascinated by the workshop module but quickly give up once they see all the options and the way that settings and results are displayed.
Good luck for your programming efforts, if you need a look at a prototype by an interested user, let me know. By the way, where do I find the mind map that Ray is talking about?
Best regards
Peter
Re: Yet another attempt to rewrite Workshop for 2.0
Peter,
thanks for the comment - they are useful! Let me quickly reply.
> because I found it too complicated
Well, the module is quite advanced and it is intended for teachers with at least minimal experience with the Assignment module and Gradebook. I'll try to do my best to prepare the UI as intuitive as possible. But the complexity is a tax for the power of Workshop. It is something similar to the Quiz module. There are lot of settings, too. However, after some time (and thanks to Olli's and Tim's good job!) teachers become quite familiar with it, right?
> set an appropriate time frame
The default behaviour will be manually switching phases - see the mockup of the mod_form in spec. Workshop will notice teacher something like "All students have submitted their work, you can now move to the assessment phase". There is an advance setting for more experienced teachers to move to the next phase automatically based on the scheduled deadlines.
> grading was not really transparent to me
There are some great proposals in this thread how to make the calculation clear. Still, however, automatic calculation of a grade for assessment is non-trivial. Please see the spec in the wiki to read more about it and tell me if it makes sense to you.
> small classess
IMO the problem is not with the number of students in a course. I had quite succesful workshop with 12 students. To reveal the real potential of the Workshop, you just need to have enough peer-assessments per submission to get the relevant grade for assessment calculated. But still, you can switch peer assessment off (to use just the grading forms) or to increase the weight of teacher's assessments so it supplies the quantity of peers (for example, set the weight of teacher's assessment to 2 and having 1 peer assessment per submission gives you 3 grades per work. Teacher's ones are considered as a reference assessment in this case. Again, see the spec for details). And also, the manual allocation of submissions for review and adjustments to random allocation will be possible.
thanks for the comment - they are useful! Let me quickly reply.
> because I found it too complicated
Well, the module is quite advanced and it is intended for teachers with at least minimal experience with the Assignment module and Gradebook. I'll try to do my best to prepare the UI as intuitive as possible. But the complexity is a tax for the power of Workshop. It is something similar to the Quiz module. There are lot of settings, too. However, after some time (and thanks to Olli's and Tim's good job!) teachers become quite familiar with it, right?
> set an appropriate time frame
The default behaviour will be manually switching phases - see the mockup of the mod_form in spec. Workshop will notice teacher something like "All students have submitted their work, you can now move to the assessment phase". There is an advance setting for more experienced teachers to move to the next phase automatically based on the scheduled deadlines.
> grading was not really transparent to me
There are some great proposals in this thread how to make the calculation clear. Still, however, automatic calculation of a grade for assessment is non-trivial. Please see the spec in the wiki to read more about it and tell me if it makes sense to you.
> small classess
IMO the problem is not with the number of students in a course. I had quite succesful workshop with 12 students. To reveal the real potential of the Workshop, you just need to have enough peer-assessments per submission to get the relevant grade for assessment calculated. But still, you can switch peer assessment off (to use just the grading forms) or to increase the weight of teacher's assessments so it supplies the quantity of peers (for example, set the weight of teacher's assessment to 2 and having 1 peer assessment per submission gives you 3 grades per work. Teacher's ones are considered as a reference assessment in this case. Again, see the spec for details). And also, the manual allocation of submissions for review and adjustments to random allocation will be possible.
Re: Yet another attempt to rewrite Workshop for 2.0
Thank you, David, for taking this project on. I have really enjoyed the Workshop module and find it very useful, but have also wished for some of the things that you describe here (in particular the clarity with grade calculations).
Re: Yet another attempt to rewrite Workshop for 2.0
Thanks Aaron.
I suppose the biggest problem to understand the Workshop grading is how the particular grade for assessment is computed. The rest should be pretty clear as the final grade is just a sum of grade for submission (which in turn is the average from all peer assessments) and grade for assessments (the average of all grades for assessments). Weight of teacher assessment can be easily understood - just imagine there are several reviewers giving the same grade (so weight 2 causes that the assessment is counted twice).
I am not going to change how the grade for assessment (also known as grading grade) is calculated. I'll try, however, to describe and visualise it somehow so teacher can possibly explain it to students (or, at least they can pretend they actually understand it kidding).
If anybody reading this thread would find a minute to read the spec part regarding the calculation, it would help. TIA
I suppose the biggest problem to understand the Workshop grading is how the particular grade for assessment is computed. The rest should be pretty clear as the final grade is just a sum of grade for submission (which in turn is the average from all peer assessments) and grade for assessments (the average of all grades for assessments). Weight of teacher assessment can be easily understood - just imagine there are several reviewers giving the same grade (so weight 2 causes that the assessment is counted twice).
I am not going to change how the grade for assessment (also known as grading grade) is calculated. I'll try, however, to describe and visualise it somehow so teacher can possibly explain it to students (or, at least they can pretend they actually understand it
kidding).If anybody reading this thread would find a minute to read the spec part regarding the calculation, it would help. TIA
Re: Yet another attempt to rewrite Workshop for 2.0
This above posting was 2009, over 4 years ago! I have spent days and hours, trying to find answers related to the Moodle Workshop. By trial and error, I have implemented Workshop in real courses, with aggregated grading. The assignment (a poster, with two aspects): 1) the poster itself worth 20 points, and 2) Citations, worth 5 points. I finally understand that each 'aspect' grade is converted to %/100. The aspects had equal weights, I eventually was able to track and tally how Moodle calculated the submission grade. However, there is no sensible explanation for how Moodle calculates the assessment grade, and I certainly can not find a coherrent explanation to share with colleagues or students. Indeed, I have found nothing later than 2009 in this matter. Well, perhaps a few bits and pieces, but from links to tech support at various universities, such that the tech support is possible only for insiders at the specific universities. It took about two months to get hypothetical students in a hypothetical course so that I could attempt Workshop before implementing it in a real course. The main priority for me is the calcualtion of the assessment grade (grading grade). I have found this page
http://docs.moodle.org/dev/Workshop_2.0_specification.
However the html text formatting and syntax of the formulas for the assessment grade need more punctuation to set of the hierarchy steps, what goes with what, for me. For example, please translate:
$sumdiffs = 0;
$sumweights = 0;
for each assessment dimension
$sumdiffs += ((bestgrade - peergrade) * dimensionweight / maxpossiblescore) ^ 2;
$sumweights += dimensionweight; Please explain $... and +,
In addition, the inconsistency of Moodle language is a huge, time-consuming, nuisance. For example, (by the way, I deplore the slopply formatting within this html or google.doc style of word processing, which adds extra spacing for line breaks - I would prefer a visually pleasing and logical formatting in an uploaded document, oh I see, a box below this box, for attachments...) What is the syntax and meaning for symbols in the excerpt below. WHAT IS DIMENSION below? Do you mean Aspects, in Workshop? Or something else?
$variance = 0
for each assessment dimension
if stdev > 0.05 then $variance += ((mean - grade) * (weight / stdev))^ 2
The file I want to attach below is too big. What I need to see can not likely fit in one of these boxes. Is there a course, or place to get answers for 2.4.4. This box refers to 2.0! Maybe I'll try another post, with pasted excerpts, but then, when I paste them into this type of box, the formatting is lost to tml or whatever is formatting within this box, such that it is extremely visually unappealing.