Currently, The only grading evaluation method is "best"paper based on a computer calculation. Where there is a teacher assessment, the best paper the computer doesn't recognize it as such and you end up having to do multiple manipulations of the weight. Because you don't want to overweight it. Is it possible to (1) be able to identify the teacher assessments and (2) compare student's assessment to teacher's assessment?
Please Vote: MDL-35722 - Allow teachers assessment to be "best" paper
by Vernellia Randall -
Number of replies: 15
In reply to Vernellia Randall
Re: Please Vote: MDL-35722 - Allow teachers assessment to be "best" paper
by Tim Hunt -
I can't let that go without commenting. The teacher may thing there evaluation is best, but is it? And, while, for each individual student assessor, the teacher's grading many be "best" (however we define that), I think it is quite likely that the average of the teacher's assessment and all the student assessments is "better" than the teacher's evaluation alone.
Of course, you are just asking for this as a option, so that the teacher can make this sort of pedagogic choice - it is quite right that the choice should be in the hands of teachers. But again, we might question even that. Is it right that an alcoholic can walk into supermarket and buy a bottle of vodka? Is it right to give the over-controlling teacher the ability to declare their grading "best"? Am I just being inflamatory? I will shut up now
In reply to Tim Hunt
Re: Please Vote: MDL-35722 - Allow teachers assessment to be "best" paper
by Mary Cooch -
Yes you are being inflammatory; I could write an essay on the subject -but it is past my bedtime so I too will shut up!
In reply to Tim Hunt
Re: Please Vote: MDL-35722 - Allow teachers assessment to be "best" paper
by Vernellia Randall -
Tim, I teach law students who have no or very little skills in peer assessment. I am teaching them a new skill legal analysis, legal issue spotting something they have no experience with.
Initially their assessments are VERY BAD and are not accurate in any since of the word. As the semester proceed and they develop their skills, I do back out of the process. BUT I think that your rant, ignores that we teach all kinds of skills, ability and knowledge. In some instances, the students assessment "may" be the best assessment in others it "may" not and the only person who can make that judgement.
And as far as the alcoholic, yes it is right to allow them buy a bottle of vodoka? Allow the smoker to buy cigarettes? Allow the fat person to buy candy?
In the end not you or anyoneelse can decide how I should best teach my class.
open source software should be designed to accomodate the why range of choices
In reply to Vernellia Randall
Re: Please Vote: MDL-35722 - Allow teachers assessment to be "best" paper
by Tim Hunt -
Thank you for explaning Vernellia. That makes the case for your feature request really clear, although as David says, it somewhat goes against the spirit of the Workshop module. You've got my vote.
In reply to Tim Hunt
Re: Please Vote: MDL-35722 - Allow teachers assessment to be "best" paper
by Derek Chirnside -
Who are you and what have you done with Tim? Is this the same Tim who told me off a few days ago? http://moodle.org/mod/forum/discuss.php?d=212322#p925618
"If you can't say anything good, don't say it at all?" and "there is always someone more helpful and patient than yourself who will respond" ? (and we did think of mary of course)
In reply to Derek Chirnside
Re: Please Vote: MDL-35722 - Allow teachers assessment to be "best" paper
by Tim Hunt -
I hope my post does not merit your criticism.
I was genuinely intersted in the philosophical and pedagoic issues raised by Vernellia's feature request. On the one had, moodle likes to try to promote a social constructivist pedagogy, and the workshop is one of the best tools for that; on the other hand, as has now been clearly explained, there are situation were you need to maintain the "them and us" separation between teacher and student, even in the workshop.
So, I do not apologies for questioning these issues, even if they way I asked the questions could have been done better. Fundamentally, Asking questions is a powerful technique to try to promote learning in a social constructivist setting - at least that is what I was trying to do, and I certainly learned from Vernellia's answer, so I thank her for taking the time to explain to me.
In reply to Tim Hunt
Re: Please Vote: MDL-35722 - Allow teachers assessment to be "best" paper
by ben reynolds -
I've been working my way through the emailed version of this conversation in reverse chronological order. The horrible rant I was anticipating from Tim turned out not to be cringe-inducing nor even what I would call a rant.
I'm now off to think about Vernellia's situation, which does make me ponder about power . . .
In reply to Vernellia Randall
Re: Please Vote: MDL-35722 - Allow teachers assessment to be "best" paper
by David Mudrák -
Hi Vernellia. Thanks for the suggestion. I am a little bit afraid that such feature would go against some basic concepts of the Workshop module. Although the use case you describe was considered during the design of the module. And I believe the solution is easy - just raise the weight of the teacher's assessment. If it has, say, three times higher weight compared to peers' weight, its impact will be three times more significant - and not only for the the calculation of the grade for submission but also for the grade of assessments.
As has been explained in another thread in this forum, if some assessment has higher weight, the "best" evaluation method considers the situation as if there were three independent reviewers who all gave the same assessment. Therefore, such an assessment is very likely to be picked as the "best" and used as a reference for the others. In other words, there would have to be at least three students (with default weight), all giving the same assessment (but different to the teacher's one) to "vote down" the teacher's assessment.
Of course, there is still this unresolved issue MDL-26794 that would simply allow you to set the higher default weight for a certain role(s) like teachers. I believe that once MDL-26794 is implemented, it will address MDL-35722 as well.
In reply to David Mudrák
Re: Please Vote: MDL-35722 - Allow teachers assessment to be "best" paper
by Vernellia Randall -
Look at my post david. When you are teaching a new skill, that new skill being how to assess complex legal documents. Do you really want three inexperience students to vote down the teachers "Correct" assessment? Furthermore, for reasons I don't understand I usually have to weight the assessments much more than "3". Students are NOT three independent
reviewers - they are students-- they are learning skills, knowledge and abilities and they are assessments can be way off. We are one week into the semesters, students are doing two assessments a week and they are still not at a point where I feel comfortable allowing their assessment to stand as indpendent reviewers. I think the basic concept that doesn't allow a teacher to identify their answer as best is a flawed concept. It should be an option for the very many circumstances in which workshop is used.
revi
In reply to David Mudrák
Re: Please Vote: MDL-35722 - Allow teachers assessment to be "best" paper
by Vernellia Randall -
I work around the problem now. By weighting my assessments enough so that it is the "best" and is used for submission grades and assessments grades. The problem is that the weight is not the same for every student. Currently, I have to
(1) go into all 60 problems change the weight, I usually use 5 and
(2) then go back into the ones where the weight is not sufficient;
The unresolved issue MDL-26794 will fix (1) but it won't fixed (2)
Allowing the teacher to designate the best paper would be the better option.
In reply to Vernellia Randall
Re: Please Vote: MDL-35722 - Allow teachers assessment to be "best" paper
by David Mudrák -
Right. The easiest way to implement this feature request seems to be via a new grading evaluation subplugin. Such a new method would be similar to the current "Comparison with the best assessment" - it would just allow to explicitly choose some users and it would consider these users' assessment as referential ones. Clean and doable.
p.s. Please note I am not quite sure what the attached screenshot above should illustrate. If I understand it correctly, it just supports my suggestion mentioned earlier - just give the teacher higher weight and it is likely to be picked as the best. Then go thru the cases when even the higher weight was not enough - maybe there was some misunderstanding in the assessment instructions? Why ALL the reviewers assessed the submission in this way? I still believe this sort of analysis has potential to provide very valuable feedback for the teacher.
p.s. Please note I am not quite sure what the attached screenshot above should illustrate. If I understand it correctly, it just supports my suggestion mentioned earlier - just give the teacher higher weight and it is likely to be picked as the best. Then go thru the cases when even the higher weight was not enough - maybe there was some misunderstanding in the assessment instructions? Why ALL the reviewers assessed the submission in this way? I still believe this sort of analysis has potential to provide very valuable feedback for the teacher.
In reply to David Mudrák
Re: Please Vote: MDL-35722 - Allow teachers assessment to be "best" paper
by Colin Fraser -
mmm This may be ok for adults and senior school students, but only, I suspect, if they have grown into an environment of sudent centred or peer assessments. Every time I have tried to do this, I have had very few positive responses, even on simple assignments using clear language rubrics as an assessment guide. Friends do well, non-friends don't, usually. I know this is a tested assessment process, but either I am doing it badly or the kids I get are so disinterested they cannot be bothered.
The worst is that I can see where it can be claimed that a teacher's assessment is not always "the best" assessment possible. I do not consider the names on any piece of work until after I mark it. For a new class, this often means that more than one student will tell me that they have never received an "A" or a "B" for anything they have handed up before. (Amazingly, this has other implications, and more than once I have had supervisors ask me if this student has done that well, and are amazed at the quality of the student's work.) This is why I am also eager to to get peer assessment right, to find that elusive, for me, formula that can allow a more co-operative culture to grow.
Any tool that can allow me to do that will be welcome.
In reply to Colin Fraser
Re: Please Vote: MDL-35722 - Allow teachers assessment to be "best" paper
by Vernellia Randall -
Colin, my experience is that even adults who have graduated from college have very little experience doing peer and self assessment. And they have a tendency to want to just give all good, all bad or straight down the middle. So for the students to get the best learning, I spend a good part of the semester teaching them how to be assessor. Here is what I have established with :
- Have the assessment grade weighted more than the submission grade until I am confortable that most students assessment skills are at least average. It takes time, I am 4 weeks into the semester and student assessment skills as a group are improving, but they are not where they need to be to.
- Use accumulative grading instead of rubric. I still set-up the grading assessment form as a kind of a rubric but I use the accumulative grading because it has a place for the students to comment. The students are required to explain on each element why they graded the element the particular way. Thus making them more conscious of the assessment process.
This is how I am currently approaching my students, it seems to work well. Generally, by the middle of the semester, most students are where I want them to be.
In reply to Vernellia Randall
Re: Please Vote: MDL-35722 - Allow teachers assessment to be "best" paper
by Colin Fraser -
Thank you Vernillia, they are great suggestions. "...making them more conscious of the assessment process." is a much better approach. Asking them to be deliberately aware of why they make a particular decision will be a thought provoking exercise. I have used the line to one group of students "It is not enough to say "It sucks", you have to be able to say why it sucks." so I might have been on the right track if I had said it before they made the assessments.
In reply to David Mudrák
Re: Please Vote: MDL-35722 - Allow teachers assessment to be "best" paper
by Abdenour Saaid -
Hi David.
Firstly thank you for the outstanding work on the workshop and i am sure it will develop into a more powerful tool in the future.
I have just a request to make with regards to the integration of a full html editor as part of the workshop assessment phase.
One of our teachers pointed out that he usually make comments and collor sections of the students' assessment with different colors and use quit a bit of formating when giving his feedbacks to students.
At the moment the workshop (submission phase) does not show the full tiny-html version.