The Workshop Module Reloaded: Version 2004081100

The Workshop Module Reloaded: Version 2004081100

by Ray Kingdon -
Number of replies: 27
This is a major update to the Workshop module. The main changes are:

  • Dropping of the Calculation of Grades stage;
  • The teacher is not asked to grade assessments (although the grading of submissions by the teacher is still recommended);
  • Grades are available from the start of the workshop and are updated continuously;
  • Students are allowed to re-assess the example submissions and the peer submissions;
  • Flexibility on how assessments are compared;
  • Improved handling of teacher assessments and their weighting in the final grades.
Overall the module has been simplified and made easier to run. The module still gives grades to assessments and these can contribute to the students' overall grades. The grading of the assessments is now automatic and goes on in the background (via the cron routine). The module has a new option which controls the comparison of assessments. This option can be tuned to set the "level" of grades given to assessments. Further, assessments which are so poor that their grade is zero are dropped when calculating the grade of submissions.

The module is now far less time critical. Although the number of phases/stages has only been reduced from six to five, a workshop can be safely left in the Submission/Assessment phase and it will produce sensible grades for the students. (In the old module the workshop had to be closed before grades could be produced.) Thus this version will allow "rolling" workshops which just carry on.

All good news then big grin. The only problem is there's a lot of changes here so please treat this version as experimental. If you do download it from the Modules page, please login as admin to get the database updated. It would be good to get it stable for 1.5.
Ray


Average of ratings: -
In reply to Ray Kingdon

Re: The Workshop Module Reloaded: Version 2004081100

by Przemyslaw Stencel -

First of all, thank you for working on this module, Ray. Although I did find it a little daunting at first, now I think it is just great! And getting better al the time, as we can see big grin.

I have two questions about your recent changes:

1/

Students are allowed to re-assess the example submissions and the peer submissions and the Calculation of Grades stage has been dropped.

This caters beautifully for the "rolling" type of workshop, but how will it work in the case of "deadlined" workshops? I mean, how will the teacher be able to put a stop to further re-assessments and finish the workshop in time-sensitive courses?

2/

You write that the teacher is not asked to grade assessments, but then you write that the module still gives grades to assessments. I'm confused - will students' assessments be graded or not?

Average of ratings: -
In reply to Przemyslaw Stencel

Re: The Workshop Module Reloaded: Version 2004081100

by Ray Kingdon -
Thanks Przemyslaw, to answer your queries

  1. Workshops can still be deadlined. the stages are still there and basically moving the workshop to the final stage (now stage 5) closes it as far as new submissions and assessments. All the students can do is to review their grades (in a little detail) and wonder why they didn't try that bit harder (well, one can always hope...).
  2. Ah, yes, there are grades given to assessments and they can count towards the student's overall mark. Basically the module looks at each submission, calculates the mean of its assessments, works out which assessment is closest to that mean then compares all the other assessments to that "best" assessment. The comparison works out a mean square distance which gets converted to grade. There I said it, bet you wished you hadn't asked big grin. Anyway, this grading takes place in the cron job and there's a new option which allows the teacher to vary the comparison of assessments from "Very Lax" to "Very Strict". Very lax comparisons will give higher grades to assessments than Very Strict. The setting can be changed in the fly so the teacher can "tune" these grades to level that they feel is right. I think it will work in practise...
Ray
Average of ratings: -
In reply to Ray Kingdon

Re: The Workshop Module Reloaded: Version 2004081100

by Przemyslaw Stencel -

bet you wished you hadn't asked

blushshyshywink

Thanks for explaining it... again. big grin

Average of ratings: -
In reply to Przemyslaw Stencel

Re: The Workshop Module Reloaded: Version 2004081100

by Ray Kingdon -
No problem. I forgot to add the new help files for this version yesterday. I've just put the five files into CVS. They should be in the downloads tomorrow. Without that help these new features would be truely baffling big grin.

Oh, the other thing is the module now understands groups. Well, it understands SEPARATEGROUPS. I think the code just needs a few minor nips and tucks and it will work with VISIBLEGROUPS as well, expect that in a couple of days time.

Average of ratings: -
In reply to Ray Kingdon

Re: The Workshop Module Reloaded: Version 2004081100

by Riaan Bredenkamp -
On behalf of people who will be using Workshop as a "rolling" activity - thanks. Makes life much easier than switching between stages all the time. Can't wait to try it out in the real world. Our stie will go live in about a month's time. Having prelimanry tests this weekend.

Riaan
Average of ratings: -
In reply to Ray Kingdon

Can teacher grade student comments?

by Gustav W Delius -
Hi Ray,
your module will feature prominently in my teaching next term. I am therefore now starting to familiarize myself with the new version. I hope you don't mind me posting many stupid questions here.
The first one: in the help file on "Grading Strategy" it says "The teacher, however, can, if desired, grade the student comments". Is this still true? Discussions above led me to believe otherwise. Does that help file need updating? Similarly in the help file for "Agreement of Assessments" it says "There may be feedback from the teacher if the teacher chooses  to grade the student assessment".
Average of ratings: -
In reply to Gustav W Delius

Re: Can teacher grade student comments?

by Ray Kingdon -
First point. Ah, yes I saw that in the help file and possibly wrongly decided to leave it stand. The exercise module certainly allows the teacher to go in and change the "grading" grades (these grades are now calculated automatically). The workshop module should have a similar facility (via the Admin page). It's a case of the coding not being up to speed with the documentation I'm afraid sad.
Second Point. Ah, that's probably the help file being out of line this time. There is/was a field in assessments which is called Teacher's comments. It used to allow teachers to add comments on an assessment. I'm now of the opinion that's too subtle. The student is firstly interested in the grade on the their work, they might be interested in comments on their work, they could be interested in getting good grades for their assesments but they wouldn't be interested in getting comments on their assessments. Further, from the teacher's point of view there's an awful lot of the assessments and they're very difficult to compare and comment on. So in the new version of workshop that field is obsolete and will probably be dropped in a future version. I've changed that help file, it's a bit simplier now and contains nothing about teachers grading assessments smile.
Average of ratings: -
In reply to Ray Kingdon

Re: The Workshop Module Reloaded: Version 2004081100

by Wendi Dunlap -
Oh, thank you for the improvements to make "rolling" workshops possible! Now we might be able to use the workshop module on my site after all! big grin (It just didn't work for us at all, the old way.)
Average of ratings: -
In reply to Ray Kingdon

Workshop module in 1.4 vs 1.5

by Gustav W Delius -
I am a bit confused now. Is the workshop module included in Moodle 1.4 supposed to be this new one or the old one? It appears to be a bit of a hybrid.
Average of ratings: -
In reply to Gustav W Delius

Re: Workshop module in 1.4 vs 1.5

by Ray Kingdon -
Ah, I think it's a hybrid. The version in 1.4 is I think the old version of workshop. At least, I have not "back-pedelled" the new version into the 1.4 branch. The new version was put into HEAD and thus it's called 1.5 Dev on the Modules Download page. (Lots of it is brand new and it needs a bit of a bash.)

The confusion arises I think because of the help files. As well as revamping the code I went through the help files and "removed" those not referenced by the new version (about five I think). Don't think I should have done that black eye. I think those files have now gone from all the branches not just HEAD.

Anyway, the new version of the workshop is not in 1.4 (nor in 1.4.x), it's only available from the Downloads page (1.5 dev). And I think the missing help files for the old version need putting back... Help! Eloy to the rescue?

p.s. I'm running 1.4 and then "cherry-picking" modules from the Modules page. In other words the new version of workshop will run quite happily with 1.4, there is no need to run a bang up-to-date Moodle 1.5 dev.
Average of ratings: -
In reply to Ray Kingdon

Re: Workshop module in 1.4 vs 1.5

by Gustav W Delius -

Ah, that explains a lot of my confusion. You can now ignore all those bug reports in the bug tracker.

Martin adopted this policy of merging changes in the language files of the development branch back onto the stable branch in order to make things simpler for translators. I am not so sure any more that this is such a good idea. 

The reason I expected the new workshop module to be in Moodle 1.4 was that I only looked at the version number 2004081100 and didn't notice that your announcement was dated September 7.

Average of ratings: -
In reply to Gustav W Delius

Re: Workshop module in 1.4 vs 1.5

by Ray Kingdon -
"merging changes in the language files of the developemnt branch back into the stable branch", well I'm all for trying to simplify things. I think I was just overhasty in deleting the "obsolete" help files. They can be deleted but not just yet. In a year's time they will be dust.

Ah, the version number was around the time I started the mods not the time I finished them. I also waited for 1.4 to roll out before committing the new version, it wouldn't be fair to put the new version in that release.

Average of ratings: -
In reply to Ray Kingdon

Re: Workshop module in 1.4 vs 1.5

by Martin Dougiamas -
Picture of Core developers Picture of Documentation writers Picture of Moodle HQ Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers
Yes, since language packs are so extensive and slow-moving, and are usually the first thing every new admin has to install, I want to keep things simple so that any given language pack works on any version of Moodle.  Branching the language packs will lead to nightmares for everyone.

If we need to make a new version of a help file, just call it filename2.html as we discussed, and keep all the old files around.  Especially in the 'en' pack.

The index pages may not always work when you make major overhauls but that's not too bad ... we just have to try our best to "grow" it slowly rather than chopping and changing stuff around.
Average of ratings: -
In reply to Martin Dougiamas

Re: Workshop module in 1.4 vs 1.5

by Ray Kingdon -
<blush> I've put the five "live" help files back into lang/en/help/workshop. I hope they've gone into the MOODLE_14_STABLE. If you can check... One of the files is called analysisofassessments.html, it should be in 14_STABLE but not in HEAD. (Perhaps they should have gone straight into HEAD without any fiddling about but I'm sorta convinced they're not needed in HEAD.) </blush>
Average of ratings: -
In reply to Ray Kingdon

Re: Workshop module in 1.4 vs 1.5

by Martin Dougiamas -
Picture of Core developers Picture of Documentation writers Picture of Moodle HQ Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers
Thanks, Ray, but please just put them back in HEAD... the downloadable language packs are the ones from HEAD so all the files should be in there.

Don't worry about STABLE ... I merge the latest packs over there periodically.
Average of ratings: -
In reply to Martin Dougiamas

Re: Workshop module in 1.4 vs 1.5

by Ray Kingdon -
OK, I've now put the little evil's into HEAD. Sorry I ever deleted them now... smile
Average of ratings: -
In reply to Ray Kingdon

Re: Workshop module in 1.4 vs 1.5

by Ger Tielemans -
is this also true for your other children like exercise and lesson? (take 1.5..)
Average of ratings: -
In reply to Ger Tielemans

Re: Workshop module in 1.4 vs 1.5

by Ray Kingdon -
Certainly, I've taken the stable 1.4 "core" and added the latest versions (1.5 dev) of those modules. The modules check the version and if it's compatible they install. If the "core" is too old they don't.

Average of ratings: -
In reply to Ray Kingdon

Re: Workshop module in 1.4 vs 1.5

by Ray Kingdon -
Ah, I'm afraid we've got a case of Sod's law. The new upload routine in this module (and others) now means that they currently fall over when run in Moodle 1.4. Their "Requires" version number need to be reset (or the new upload/virus checking routines back-pedelled into 1.4.x)

It shows the strength of the versioning system in Moodle that we can do these things (even if we may not always do them straight off wink ).
Average of ratings: -
In reply to Ray Kingdon

Re: The Workshop Module Reloaded: Version 2004081100

by Mark Pearson -
Ray,

We're currently running Moodle v 1.3.3 at Earlham (2004052503) and don't plan on upgrading until the end of the semester. I tried to use the Workshop module in a live class setting (which failed I'm afraid) and I found it rather cumbersome and flakey - I can post details but it seems pointless when there's a new improved version available. I'm wondering whether the new veriosn of the workshop module will run with Moodle 1.3.3 and whether postgres is fully supported as a back end database.

Thanks for all the effort on HTML Area - I had to have my students copy their answers from a Journal into Dreamweaver and save as a file to upload. A messy process and one which lost all HTML formatting!

Average of ratings: -
In reply to Mark Pearson

Re: The Workshop Module Reloaded: Version 2004081100

by Ray Kingdon -
Mark, the problem with the Workshop module is the flexibility. There's lots of different ways of using it. I'm rather stuck in a rut locally and use it in one way most of the time. Not surprisingly it works fine for me, but that doesn't help, I know.

The upload process has recently been changed radically in the new version. It can now do virus checking on the student's files. (Thanks to Penny.) This certainly stops it being run with older versions of Moodle (at the moment).

The HTML area work is not mine, I just make use of what other's have done, it's a really useful tool now.

On that subject I'm currently working on the module so that students can write their submissions directly into the HTML editor. The submissions can then be totally within Moodle. The submission of work in "external" formats (for example Word files) will still be there. But that will be done through the paradigm of attachments. If the assignment requires students to submit files it will allow submissions to have an arbitrary number of attachments. Most assignments which require external files will, I image, require just one attachment. But I have an assignment in mind where two attachments might be the norm. So an arbitrary number of attachments makes things flexible. (Ah, the f... word againbig grin.) It's only one extra option to the (already long) list sad.

Ray
Average of ratings: -
In reply to Ray Kingdon

Re: The Workshop Module Reloaded: Version 2004081100

by N Hansen -
Ray-I'm having a bit of trouble following the discussion of where the help files are and if they have been updated to reflect the changes you have made, so forgive me for asking this question if it is answered in the help files. With the changes you have made, would it be possible to have a "rolling" workshop without a specific deadline whereby once a student had submitted their work and it had been assessed by the teacher and/or other students, the student would be allowed to see the league table, but not before? If not, I would like to see such a feature. I'm going to be teaching a self-paced course that students can enroll in at any time, but being able to see the work and assessments thereof of other students' work who came before them after submitting their own (to prevent copying) would be very beneficial to my students. 
Average of ratings: -
In reply to N Hansen

Re: The Workshop Module Reloaded: Version 2004081100

by Ray Kingdon -
The display of the League Table of the best pieces of works does, at the moment, only happen if you "close down" the workshop. It's a feature of the last phase of the workshop.

However, a similar review of work previously submitted by other students is acheived by simply making the students do some peer assessments. So if you set the option "Number of Assessments of Student Submissions" to say, 5, then after a student submits their own work they are presented with 5 pieces of work from the other students. The student is asked to assess these and can be given grades for these (peer) assessments. In the latest version of the Workshop module the "better" these assessments are, the higher grades. ("Better" here usually means in close agreement with the teacher's assessment, but it can also mean close agreement with how other students have assessed that particular piece of work. Thus the teacher does not have to assess every single piece of work...).

The way the work is allocated to the students for assessment means that they usually given recent submissions. Thus the new students are unlikely to see too many "old" submissions. A "rolling" workshop with peer assessments should mean that mostly the current cohort of students get their own pieces of work to peer assess.

Hope this helps

Average of ratings: -
In reply to Ray Kingdon

Re: The Workshop Module Reloaded: Version 2004081100

by N Hansen -
OK, that makes sense, but it leads to another question. If a workshop were set up so only the teacher were allowed to make assessments, could students still be allowed to see a certain number of other students' work, without making assessments of them? You see, for what I want to do, it wouldn't make much sense for students to be assessing one another, but it would be beneficial for them to see their fellow students' work and the teacher's assessment of that work.
Average of ratings: -
In reply to N Hansen

Re: The Workshop Module Reloaded: Version 2004081100

by Ray Kingdon -
Yes, the alternative approach might be more sensible. There is an option to show a number of examples to the students before they can submit their own work. It's the "Number of Assessments of Examples from Teacher" option. If you set that to 5, say, the students are shown 5 pieces of work which they must assess before they can do anything. The teacher submits a number of examples assignments into the workshop, say 10 pieces, so the students get different things to look at. (It's done in the first phase, in a "rolling" workshop the workshop would be briefly put into phase 1, new examples submitted and the workshop to phase 3.) These examples should be assessed by the teacher. The students are not shown these assessments but they shown how close their assessment of an example is to that of the teacher's. If it's not that close they get the chance to do the assessment again.

As I say you can add to the pool of examples assignments at any time (by briefly "rewinding" the workshop to phase 1). The allocation of these examples to the students is similar to that for the peer assessments. The examples which have the fewest "appearances" are the ones that shown. So the students are likely to see the new examples first.

This, as I say, is an alternative to peer assessments. It's still requires the students to do some assessments but it more controlled because they are looking at the work you choose to submit and their assessments are benchmarked against your assessments. I know it asks the students to do assessments but without them I not sure the students would look at the work...

Average of ratings: -
In reply to Ray Kingdon

Re: The Workshop Module Reloaded: Version 2004081100

by N Hansen -
I don't think I made myself clear. I do not want students to see the work of their fellow students before they complete the assignment, because it is a translation exercise and that is the kind of assignment where they could just copy the work of others, but I would like them to see the work of others after they complete it. I would also like the students to be able to view the assessment the teacher made of their fellow students' work, and it would be preferrable if the teacher could make that assessment directly in the work submitted with the html editor itself. I also need it to show them the other students' work immediately after submitting their own work, or at least after the teacher has assessed it, because the course is self-paced with rolling enrollment so there cannot be a phase where the workshop is closed.
Average of ratings: -
In reply to N Hansen

Re: The Workshop Module Reloaded: Version 2004081100

by Ray Kingdon -
if you want to show the students work before they submit you set up examples; if it's after they submit then it's by peer assessments. Those are the two alternatives available through the Workshop module. smile
Average of ratings: -