Editing questions

Editing questions

Gustav W Delius -
回帖数：47

Currently when a question is edited it is quietly changed in all quizzes that are using it, even if the quizzes are already open. I think that is dangerous. It can lead to confusion and to justified complaints from students. I propose the following:

If a teacher edits a question that is already being used in a quiz then the a copy of the question is made and this copy is edited. The original version is kept intact. The teacher is then asked for each quiz in which the old version of the question is already used whether or not to replace the old version by the new version. If the teacher chooses to replace the old version and there are already student attempts at the old version then the teacher is asked whether or not to apply the old responses to the new version and re-mark.

平均分： -
回复Gustav W Delius

Re: Editing questions

Ray Lawrence -
This sounds fine, but wouldn't it need some way of producing a historical list for audit purposes? How would someone establish what the changes were and when they occured at a future date?


平均分： -
回复Ray Lawrence

Re: Editing questions

Gustav W Delius -
My proposals were modest and only aim to avoid accidental overwriting of questions. Always keeping past versions for audit purposes would be a good idea, the only reason I didn't propose it is that it is a bit more work to implement.
平均分： -
回复Gustav W Delius

Re: Editing questions

Gustav W Delius -

I would like to elaborate more on my proposal on what to do with older versions of a question. As I wrote earlier, when a teacher edits a question that is already in use in one or more quizzes then the teacher should be alerted to the fact and should be given the choice for each quiz of either keeping the quiz in its old state or replacing the question by its new version. In the later case the teacher should also be asked whether existing student responses should be re-marked.

In either case it will be necessary for Moodle to keep a copy of the question in its original version. Even if the teacher decided to replace the old version by the new version a copy of the old version should be kept around so that it is later possible to reconstruct which question the student actually saw when he or she entered the response. The old versions which are only kept for this purpose and should not be used for new attempts have to be marked as such. I therefore propose an extra field 'newversionid' in the quiz table which holds the id of the newer version. If this field is zero then there is no newer version. If the field is non-zero then Moodle knows that this question is only there because it is already being used in some quizzes but it should not be shown in the list of questions on the quiz edit page.

As usual the random questions complicate matters. We need to avoid that a random question chooses a question of which an old version is already used in the quiz. Given the information in the 'newversionid' field this can be done.

Did I forget anything?

平均分： -
回复Gustav W Delius

Re: Editing questions

Julian Sedding -
I have discussed a possible approach to editing questions in a more sensible way with Gustav and we came up with a solution, that keeps track of the question versions for each quiz in a separate DB table. So if an existing question (that already wass attempted) is edited a copy with the changes is automatically made and an entry of the format |quiz_id, old_question, new_question| is made.

For the user interface we thought that it is important to show in which quizzes the edited question is used and by how many students it has been attempted in every quiz, because that gives a good indication of the impact of any changes made. However, because it is possible that the changes should only affect one quiz, the teachere will be allowed to decide in whch quizzes the old version should be kept/replaced.

I have made a mockup of the illustrating default values I thought were sensible. Please feel free to comment on this as I am just starting to implement this. The checkboxes shown will go onto the editing page of every question.
平均分： -
回复Julian Sedding

Re: Editing questions

Dawn Wright -
Julian, I have reviewed the mockup screen and I think it will work well. When will the screen be displayed? I can see utility in knowing where a question has been attempted before I make changes and also having an opportunity to decide where any changes I made will be effective. I guess it would be possible to "re-edit" a question just to change the impacted quizzes via the check boxes on this screen. 
平均分： -
回复Dawn Wright

Re: Editing questions

Julian Sedding -
Hi Dawn,

thanks for your comments! The edit screen will be displayed on the editing page of the question. I agree it would be nice to have a "re-edit" functionality. I propose that that it would be along the lines of "There is later version of this question available, do you want to use it for this quiz?" along with a preview link to the newer version. Does that sound acceptable?

Julian
平均分： -
回复Julian Sedding

Re: Editing questions

Gustav W Delius -
Dawn and Julian,

I am not sure you are both talking about the same "re-edit" functionality. I have the feeling that Dawn is talking about editing an older version again while Julian is talking about an easy way to fetch newer versions of a question into quizzes that were originally planning to keep the old version.

In any case I think such bells and whistles can wait until later.
平均分： -
回复Gustav W Delius

Re: Editing questions

Dawn Wright -

Gustav, I was referring to when the information about where the question has been used would be visible. I think I would prefer to know where the question is being used before I edit it rather than after.

平均分： -
回复Dawn Wright

Re: Editing questions

Gustav W Delius -
Yes, that information will be available on the page on which you edit the question. 
平均分： -
回复Gustav W Delius

Re: Editing questions

Julian Sedding -
For now I have put the information below the save question button on the editing page. That way the new table (which as Martin pointed out can be quite long) doesn't hinder the editor and can even be ignored if the defaults are good enough.
平均分： -
回复Julian Sedding

Re: Editing questions

Martin Dougiamas -
Core developers的头像 Documentation writers的头像 Moodle HQ的头像 Particularly helpful Moodlers的头像 Plugin developers的头像 Testers的头像
The interface looks good to me (with this being shown on the question editing page).  满意  I guess the list might get long with some of the sites I've seen that re-use question-banks a lot.

From a back-end perspective this needs to be thought through carefully ... I can sense the possibility of problems during backups and when sharing categories to other courses.  Also will the quiz-editing interface be burdened/confused  with multiplying copies? ... none of it is insurmountable but this needs to be thought out before starting ...
平均分： -
回复Martin Dougiamas

Re: Editing questions

Julian Sedding -
Hi Martin,

I don't know much about how the backup feature works. Could you please point me to where you think problems could occur?

The problem with shared categories would be if the "Old Questions" category wasn't shared, I guess. That is a problem that needs some consideration indeed. Thanks for pointing it out!

To avoid cluttering the quiz-editing interface the duplicate copies are moved to the "Old Questions" category. So the number of questions in the category used for a quiz doesn't change. Only if the old copy is chosen to be left in its original category will there be a problem.

Julian
平均分： -
回复Martin Dougiamas

Re: Editing shared questions

Gustav W Delius -
Yes, shared questions are indeed a problem. Currently it is very dangerous to use questions from a shared category because someone outside your course can change the questions in your quizzes, possible while or after students are answering them. At least with the new scheme the teacher changing questions in a shared category would be told if quizzes in other courses were using it.

There is one question that we need to decide: do we really want to give the owner of shared question the power to change them in other teacher's quizzes?
平均分： -
回复Gustav W Delius

Re: Editing shared questions

Dawn Wright -

Gustav

We often have six or more copies of a course active with students assigned to different instructors. Individual instructors are not allowed to edit but our ID staff often has to correct mistakes in exam questions and it would be nice to just have to do that once in the master course instead of going into each copy of the course. Having the ability to use a shared category across courses has utility for us.

dawn

平均分： -
回复Dawn Wright

Re: Editing shared questions

Gustav W Delius -
Yes, that makes sense. So owners of questions in shared categories should be able to change them in all quizzes that use them, even if they are not teachers in the course.  After all teachers who don't want that to happen to their quizzes can simply make copies of the question for their own course.
平均分： -
回复Martin Dougiamas

What to do with old versions of edited questions

Gustav W Delius -
Moving old versions of quizzes to a category "Old questions" does not look like such a good idea because this would loose the information about which category they were in originally. I think it would be better to give them a 'hidden' flag which would have the effect of not showing them in the list of active questions. Later we could implement a checkbox on the quiz edit page that lets the teacher request to see the hidden page as well.

This hidden flag requires an extra field in the quiz_questions table.
平均分： -
回复Gustav W Delius

Re: What to do with old versions of edited questions

Julian Sedding -
I agree, using a hidden-flag also gets us around the problem of shared categories, because questions still remain in their original category. So if there are no objections I will implement it that way. To later show all or only the hidden questions should not be a problem.

On an other account: at the moment, how is the case handled where a teacher moves a question from a shared category to a private one? Does that break the quizzes of peple using the shared question? Or is this conflict resolved somehow?
平均分： -
回复Gustav W Delius

Deleting questions

Gustav W Delius -
As was just pointed out by Eloy in a different thread, this issue also arises when a question is deleted even though it already had attempts. In this case too the old version of the question needs to be kept. Perhaps the 'newversionid' field should be set to -1 for such questions?
平均分： -
回复Gustav W Delius

Re: Deleting questions

Eloy Lafuente (stronk7) -
Core developers的头像 Documentation writers的头像 Moodle HQ的头像 Particularly helpful Moodlers的头像 Peer reviewers的头像 Plugin developers的头像 Testers的头像
And, from my absolute unknowledge of how teachers use to work, wouldn't be easier to simply avoid:

- Modify and delete questions from one quiz.
- Modify and delete questions from the bank.

If it has been answered anywhere (quiz_responses).

It would mantain integrity of quizes and grades perfectly and, with the new "duplicate question" feature it's really esay to make a copy to modify and to use in other quiz.

Am I missing anything?

Ciao 微笑
平均分： -
回复Eloy Lafuente (stronk7)

Editing and deleting questions

Gustav W Delius -

If teachers were perfect then your solution would work. However in real life teachers make mistakes when setting questions and these are often discovered only after the first few students have attempted the quiz. So it is necessary to allow teachers to edit questions even though some students have taken them. Furthermore it is necessary to allow teachers to choose whether to remark students' previous responses or not.

Giving teachers the possibility to delete questions after some students have taken them may be less important, but given that we have to put the technology in place for editing we can just as well provide for deleting at little extra cost. I can think of scenarios where the teacher might want to remove a question from a quiz. Take for example the case where a question proves unexpectedly to be entirely too difficult for the first few students who try it. The teacher might then decide to discuss the question in class and remove it from the quiz.

平均分： -
回复Eloy Lafuente (stronk7)

Re: Deleting questions

Enrique Castro -
Core developers的头像
Hi Eloy,
    I see a problem for new quizzes that use random questions. Even if you create a new version of the question, ¿how could you prevent the old one been selected into a quiz as a random question?

The versioning mechanism proposed by Gustav would take account of this. But a versioning system may be difficult to maintain. Another possibility, that may have other simple uses is just a tag to "exclude this question" from random quizzes.

On the other hand, editing a question and re-grading a quiz is not as unusual as it would be is teachers were perfect machines. In practice, you may need to correct an ambiguous sentence, accept as valid an answer you didn't mark etc. So, this kind of functionality has its own target.

- Enrique -
平均分： -
回复Enrique Castro

Re: Deleting questions

Eloy Lafuente (stronk7) -
Core developers的头像 Documentation writers的头像 Moodle HQ的头像 Particularly helpful Moodlers的头像 Peer reviewers的头像 Plugin developers的头像 Testers的头像
Hi,

now that we have seen that teachers are imperfect 吐舌头, I can understand the need to be able to add, modify and delete questions from a "running" quiz.Thanks!

Anyway, I think that we should define everything (each posibility and its implications) completely before thinking in possible solutions (implementations).

As I see the problem, we are in an "attempted quiz". This is the origin of our headaches, ok?

Then we have to analyse these:

- Add question to quiz.
- Modify question in quiz.
- Delete question from quiz.
- Add question to bank.
- Modify question in bank.
- Delete question from bank.

Some of the previous can have equivalent implications but, to begin, we should analyse them separately. And some of them won't be a problem but it will be a conclusion later...

Assuming that we are going to support all those actions against our "attempted" quiz, we have to answer all these question for each of them:

- Existing grades: They can be mantained unmodified or recalculated with the new quiz configuration. Will be this decission fixed (by Moodle) or selectable (by the teacher).
- Student view: Their original attempt or the modified one (determined by the previous decission, I think).
- Locks: We have to lock parts of the quiz, some always and some based in the previous decission. As I know: regrading of old attempts, reuse of old (modified or deleted) questions. Such locks must be done at some exact level: response, quiz, course, site...
- Site perspective: All this is OK for question belonging to only one course (private categories), but what happens if the same question is being used by other quizzes/courses? We cannot assume the change site-wide.
- DB structure: What is going to be the exact representation of all this in DB? How we detect locked questions, locked grades...

And all these questions must have response for each action. Only forbiding some action can be used to answer such questions.

Only a bit of brainstorming.... ciao 微笑
平均分： -
回复Eloy Lafuente (stronk7)

本讨论区帖子已移除

本讨论区帖子的内容已移除，无法再访问。
回复删除的用户

本讨论区帖子已移除

本讨论区帖子的内容已移除，无法再访问。
回复Eloy Lafuente (stronk7)

Re: Deleting questions

Enrique Castro -
Core developers的头像
Hi Eloy,

You are right that attempted quizzes are the "problem".

Editing  questions in a  particular quiz and re-grading it is an absolute MUST.  This is the kind of functionality that adds usability to the quiz. We can beat WebCT there. Editing a question means to be able to change the correct answer, the grade assigned to each answer, and the points assigned to the question within the quiz.

Adding new questions or changing the text of the questions to an attempted quiz should be avoided. These features are not a must, and can raise unexpected problems with students claiming against "ex post facto" changes. Even deleting a wrong question is not needed if you can set its contribution to total grade as 0 points.

But all these features are very convenient when creating the quiz. They should be there while the quiz has not been done by any student.

When editing a question in the bank a check should be performed at first.
a) If the question is used in a quiz, the teacher should be warned, and asked if he wants to update the existing question or create a new one (unlinked to quiz) duplicated from the original.

b) If the question is used in a quiz, and the quiz has been attempted, the question text cannot be modified. But Moodle can provide the option to automatically create a duplicate and edit it in the bank (not the quiz).

When deleting a question in the bank, it should be allowed (with warning) even if the question is in use, EXCEPT when the quiz has been attemped. In that case a message should deny deletion and inform of the quizzes using the question.

When the teacher edits or delete a question fron the quiz list, it should be assumed that he knows what he is doing. The only restriction in this case should be if the quiz has been already attempted.

- Enrique -

平均分： -
回复Enrique Castro

Re: Editing and Deleting questions

Gustav W Delius -

I agree that adding questions to a quiz that has already been attempted should be disallowed. It is too difficult to know how to grade those students who attempted the quiz before this question was added.

Deleting questions would not pose such problems, the achieved grades could simply be deleted as well. However in practice this will not be needed so often, so I agree that we could simplify matters by disallowing deletion of questions after they have been attempted.

I disagree with Enrique when he says that the question text should be non-editable after the question has been attempted. Small corrections in the question text may be very desirable. We have to leave it up to the teacher to use this possibility responsibly. And in any case the previous version of the question has to be kept for the record so that student complaints can be resolved.

平均分： -
回复Gustav W Delius

Re: Editing and Deleting questions

Ray Lawrence -
Unfortunately, even when a tremendous amount of care has been taken to ensure questions are correct, unambiguous etc., errors do occur. In my experience the key issue in that case has been that the candidate is not disadvantaged for the shortcomings in the process.

In a relatively informal class/course quiz this can be addressed without too much difficulty. In more formal testing this is not always the case and careers and lives can be affected. In the case of dispute, and for audit purposes, the Quiz / Exam must be retained in exactly the form that the candidate saw it. Without this candidates and others will quickly loose confidence in the tool and process for setting and marking.

Therefore the principal requirement is to be able to regrade the attempt based on the valid items remaining after the suspect ones have been "voided". In this way the candidate is not disadvantaged (or at least it is harder to make the case that they have).

Where questions are subsequently edited and re-used in their amended form the Quiz / Exam must be able to be recognised as a different version.
平均分： -
回复Ray Lawrence

Re: Editing questions

Gustav W Delius -
Yes, Ray is right, it is necessary for audit purposes to be able to reconstruct the quiz exactly the way it looked when a candidate made an attempt. In order to cater for this I am planning to add an extra field 'originalquestionid' to the quiz_responses table. When a response is re-graded with a new version of a question then the 'question' field is set to the id of the new version and the 'originalquestion' field is set to the id of the original version.
平均分： -
回复Gustav W Delius

Re: Editing and Deleting questions

Ger Tielemans -

The questionnaire has a hard regime: when you choose to publish a questionnaire, you no longer can edit that questionnaire, you only can make a new copy of the test, edit that and publish it as a new questionnaire.

(You will not like it in the beginning, but no misunderstanding possible.)

  • Tests that have important consequences for the students should follow this regime,
  • Tests which you use for try-out, test-yourself, and exercises with a high "where do I stand now?" component, should have the more liberal option to change, update & refine the questions AND the feedback during use. 

平均分： -
回复Enrique Castro

Re: Deleting questions

Dennis Daniels -
Please keep in mind that teachers are constantly adding data in the feedback, changing how a question is worded, adding tips and etc.

I use the quizzes daily... the more flexibility the better IMHO.

thanks for reading!!
Denny
平均分： -
回复Dennis Daniels

Re: Deleting questions

Ray Lawrence -
Dennis,

Yes, that  may be the case for some use of Quizzes but IMO certain types of flexibility (or the inability to control access to it, which is perhaps not a Moodle issue) could lead to a loss of confidence for more formal testing if it s perceived that errors can be covered up after the event.

Hmmm.... I'm sounding a bit paranoid in this post and the one above but I have seen both the difficulties this can cause and the strategies which individuals will use to obtain a favourable outcome when the stakes are high.

Ray
平均分： -
回复Gustav W Delius

Re: Deleting questions

Gustav W Delius -
I think that the upshot of these discussions here is that it is important to allow teachers to modify questions even after students have attempted them but that it is not really necessary to allow adding or deleting questions from a quiz that students have already attempted. Allowing this would lead to more confusion than it is worth.

So I have now changed edit.php in Moodle 1.5 so that if a student has already attempted a quiz the layout becomes simpler, it just shows a single table with the list of used questions. The edit and preview icons are still there but the delete icons are gone.

My next task is to take care of editing used questions along the lines discussed in my earlier posts in this thread.
平均分： -
回复Gustav W Delius

Re: Deleting questions

Michael Penney -
it is not really necessary to allow adding or deleting questions from a quiz that students have already attempted.

Hi Gustav, I'd disagree with part of this. Being able to delete a bad question after students take a test and regrade all attempts without that question counting is a nice feature, IMO.

Either that or a way to modify points manually, but that would still be a pain if 100 or so students took a test and then the teacher realizes one of the questions was a bad one.
平均分：Useful (1)
回复Michael Penney

Re: Deleting questions

Ray Lawrence -
There's still life in this one yet!

it is not really necessary to allow adding or deleting questions from a quiz that students have already attempted.

I agree with this.

Being able to delete a bad question after students take a test and regrade all attempts without that question counting is a nice feature, IMO.

I agree with this in part but deleting a question destroys the audit trail - which in the field in which I work is completely unacceptable. This must surely be an issue elsewhere too.

However, being able to regrade using only using the valid items without having to do it manually is extremely desirable. I don't know if it's possible but could an unsatisfactory question be disregarded rather than deleted for these regrading purposes? 


平均分： -
回复Ray Lawrence

Re: Deleting questions

Gustav W Delius -

Yes, Ray, an unsatisfactory question can be disregarded simply by setting its grade to zero. I tend to agree with you that doing that is better than deleting it.

The audit trail you mention is indeed essential, which is why I will add the code necessary to keep an audit trail of edited questions. I could, as mentioned in previous posts, also keep an audit trail of deleted questions but I think the current solution of simply not allowing deletion is cleaner.

平均分： -
回复Michael Penney

Re: Deleting questions

Martin Dougiamas -
Core developers的头像 Documentation writers的头像 Moodle HQ的头像 Particularly helpful Moodlers的头像 Plugin developers的头像 Testers的头像
I agree with Michael here, Gustav, about allowing deletion. I don't think your reduction in the possibilities of the quiz module is a good idea - can you please fix it again?

I know personally of several teachers who have needed to delete a "bad" question after students have taken a quiz, then regraded the quiz to bring it all back into line.

Stopping the addition of questions makes total sense though.
平均分： -
回复Martin Dougiamas

Re: Deleting questions

Gustav W Delius -

Yes, I originally had the same feeling that the ability to delete bad questions was needed until Enrique pointed out that it would be better to simply set the grade of the "bad" question to zero. That of course is still possible. I think it will be less confusing to students to see that a question has been discounted than to see it disappear.

I will be happy to bring the delete icon back if people really think that there are circumstances where it is desirable.

平均分： -
回复Martin Dougiamas

Re: Deleting questions

Michael Penney -
A number of our faculty want the ability to change any grade manually(they can do this with Blackboard now), as well as an option for students to comment if they think a question is unfair or wrong (basically the freedom you would have with a paper test--can't do that with BB).

Certainly this could be mixed with an audit trail, but in some other institutions it might be important not to let anyone change the grade after the attempt.

Perhaps options in the quiz mod settings on the admin page could serve both the 'higher' ed folks who want complete freedom (or complete professorial power微笑 and the folks who need guaranteed integrity and audit-ability of the student's work, for better or worse?
平均分： -
回复Michael Penney

manually grading questions

Gustav W Delius -
Michael, the ability to change grades manually is certainly crucial. I actually thought you were involved in implementing this at the moment, see http://moodle.org/mod/forum/discuss.php?d=15960#78413. I will start a separate thread to discuss this further.
平均分： -
回复Gustav W Delius

Re: manually grading questions

Ger Tielemans -

I would prefer not to have the option to change a grade manually, but to add visible checkable bonus/malus points in the gradebook for each grade:

I was wondering: If you use Gustav's old gradebook,  can't you insert an extra activity with manual grading (that is where you do this +/- thing) and  combine this column with the test-grade column in a new calculated column?

平均分： -
回复Ger Tielemans

Re: manually grading questions

Michael Penney -
You can do a similar thing in gradebook_cdc by adding an extra credit item to the category and then you can give individual students bonus points on the graded item(s) in the category.

But the option to change grades manually and regrade is a big issue here at least (Cal State), I'd imagine that many faculty who have enjoyed this in Blackboard in other places will also be disappointed not to find it in Moodle.

For instance, my wife's computer crashed 3 essay questions shy of complete in a blackboard quiz. She emailed her prof her answers, he pasted them into  her Blackboard quiz and graded her, and there was no other special task for him to do to solve the situation.

This is the sort of thing faculty I know like, being able to quickly and simply recover from technology problems at their own discretion, and having their grading ability limited by the application is extremly frustrating.

Manual grading is on our list of problems (identified by our faculty) preventing more of our faculty from embracing Moodle for this reason. IMO, it should be  a setting in the mod admin section, so that institutions who need to prevent their instructors from modifying quiz grades can do so while institutions who want to offer their faculty the same freedom they have with other LMS's and paper based tests, can do so with Moodle.
平均分： -
回复Gustav W Delius

Re: manually grading questions

Michael Penney -
  I actually thought you were involved in implementing this at the moment, see
Hi Gustav, yes we would like to, but funding is getting harder to come by here (the CSU system is preparing for some pretty deep cutbacks). We're going to hunker down and work on the enterprise level management features we need to show Moodle courses as cost savers over BB courses to the bean counters mostly for the next few months (I'll post a list of these soon to be sure we aren't duplicating efforts of other teams).

So we may not get to as soon as I would like to have it伤心. Fortunatly with gradebook_cdc, quickmail, myfiles submit to assignment, etc. we've addressed most of our faculty's concerns re the BB v. Moodle issue微笑.
平均分： -