Quiz Answers Removed when Student is not in the Quiz Attempt - and it Auto Submits

Quiz Answers Removed when Student is not in the Quiz Attempt - and it Auto Submits

gan Drew Smith -
Number of replies: 15

We are having issues with quiz attempts losing all of its auto-saved answers in Moodle 2.5.3, and returning a "finished" attempt as having no answers.

To test this, we have:
1) Created a timed quiz (multiple choice and essay).
2) Enabled quiz autosave to run every minute.
3) Set the "When time expires" action to "Open attempts are submitted automatically"
4) Logged in as a student and started the timed quiz (10 min)
5) Verified quiz autosave was running/working using Google Dev. Tools
6) Opened the "Review attempt" for the student quiz in progress in the admin view, to verify work has saved thus far - the status was "In progress"
7) With one minute left on the clock, we closed the browser window where the student was taking the quiz.
8) Waited for time to expire on the quiz.
9) Refreshed the "review attempt" admin view - answers were still present with a status of "In progress"
10) Logged back in as a student, selected the quiz, and it showed a zero score (even though all answers were correct and autosaved at the time of closing the browser window.  This seems to happen if the Moodle cron runs as well (provided the student didn't log in first).
11) Refreshed the "review attempt" page in the admin view - the status was set to "Finished" but there were no answers saved. It appeared there was never anything answered.

Any thoughts why the quiz attempt, which should be submitting when the time runs out (regardless of whether the quiz is currently being viewed by the student in a window or not, from my understanding)?

Thanks for your help

 

Cyfartaledd sgoriau: -
In reply to Drew Smith

Re: Quiz Answers Removed when Student is not in the Quiz Attempt - and it Auto Submits

gan Tim Hunt -
Llun o Core developers Llun o Documentation writers Llun o Particularly helpful Moodlers Llun o Peer reviewers Llun o Plugin developers

Now you point this out, I am not surprised. This is the way I would expect the code to behave.

However, one cannot really say the code was designed that way intentionally. Until reading your post, I had not considered this particular combination of circumstances in depth.

For now, the advice is, don't do 7).

To give a bit more explanation behind the scenes: there is a difference between a save and an autosave. The way the code was written, autosave was just designed to save you if your browser crashed, and then you went back into the quiz to continue working. For your responses to actually be counted, you need to do a real save before the quiz was submitted.

Now that I have thought about this, I realise this might not be the right decision. What do people think, should we change it, so automatic submit all and finish uses any auto-saved responses there might be?

Cyfartaledd sgoriau:Useful (1)
In reply to Tim Hunt

Re: Quiz Answers Removed when Student is not in the Quiz Attempt - and it Auto Submits

gan Drew Smith -

Thanks for your quick response, Tim!

I would encourage the auto submit to save any auto-saved responses.

I agree with your assessment of not doing step #7. However we were investigating a support issue with one of our students who closed their window (perhaps intentionally - perhaps accidentally) before the quiz timer ran out. When they got back to the quiz to re-enter, their time had expired and their quiz autosubmitted with no answers. This was a re-enactment/confirmation of what they had done. 

Giving the student the benefit of the doubt, even if they answered all of these questions and legitimately did the work, it still gets cancelled out. The instructor is then faced with a dilemma of 1) reopening the quiz for the student or 2) keeping the zero for the student. Problems with option 1 is that the student has now had a chance to review the questions and multiple choice options - thus giving that student an unfair advantage (provided the test bank isn't that large, and it's not a scrambled quiz - i realize there are ways to mitigate this issue).  The problem with option 2 is the student is now penalized, even though they did the work. 

Just my 2 cents. 

Thanks again for the reply. 

Cyfartaledd sgoriau:Useful (1)
In reply to Drew Smith

Re: Quiz Answers Removed when Student is not in the Quiz Attempt - and it Auto Submits

gan Joseph Rézeau -
Llun o Core developers Llun o Particularly helpful Moodlers Llun o Plugin developers Llun o Testers Llun o Translators

Hi Drew,

Some food for thought...

Point of view #1

What was the situation with the "older technology" of ink and paper? Imagine a teacher telling the school secretary this story: "Instead of handing in his exam paper at the end of the exam session, one of my students took it home (perhaps intentionally - perhaps accidentally)." The next day, that student "found" his exam paper in his bag; he now wants to hand it to me. Should I accept to mark it? The obvious answer is "No", too late, too bad. And the same should apply to your student who closed the quiz window (intentionally or accidentally). No benefit of doubt given.

Point of view #2

Given the fact that using a computer for an exam quiz, especially in the stressful context of an exam, can cause some students to accidentally click or push a key on their keyboard. Given the fact that technology allows us to - if not prevent - at least remedy such mishaps, then if it can, it should.

Looking forward to seeing more points of view.winc

Joseph

 

Cyfartaledd sgoriau:Useful (1)
In reply to Joseph Rézeau

Re: Quiz Answers Removed when Student is not in the Quiz Attempt - and it Auto Submits

gan Drew Smith -

Hi Joseph -

The ink and paper tests are moot and do not apply in this situation - there are too many variables at work here, including internet connections, level of computer savvy of the user (you may be surprised how many adults don't know how to refresh their browser), a myriad of personal computer settings, etc. These are not in-house classes taking tests online; they are fully online courses taken by adults. Some are tech-savvy, some are tech-ignorant, and yes i see the irony of tech-ignorant students taking an online course. If this quiz was being taken in-class, then i can see the validity of your point. However in this scenario, there are too many factors. 

For your point #2, you are right - technology allows us to prevent and remedy mishaps. If it can, it should. And i think a quiz autosubmitting without answers is a mishap. Here is an opportunity to continue preventing further technological issues.  

 

Cyfartaledd sgoriau:Useful (1)
In reply to Drew Smith

Re: Quiz Answers Removed when Student is not in the Quiz Attempt - and it Auto Submits

gan Tim Hunt -
Llun o Core developers Llun o Documentation writers Llun o Particularly helpful Moodlers Llun o Peer reviewers Llun o Plugin developers

Like Joseph, I would like to see a wide range of opinions on this.

Another verion of Joseph's scenario, is in a paper an ink exam, when the student writes the answer to one question on a sheet of paper that (accidentally or otherwise) gets thrown away with a stack of rought working that the student did, but did not intend to hand in.

Actually, in OU exams, everything the student writes during the exam must be clipped together and handed in. Nothing is thrown away or allow to be taken out of the exam room by the student. Students are told to do their rough working in the answer book, and just cross through it with a single line (so it is still readable) if they don't want it marked. Useful parallel?

Cyfartaledd sgoriau:Useful (1)
In reply to Tim Hunt

Re: Quiz Answers Removed when Student is not in the Quiz Attempt - and it Auto Submits

gan Drew Smith -

Perhaps it's possible to classify a browser-submitted quiz, and notate it differently than a student-submitted quiz? Like, for example, having the state be set to "Browser Submitted" rather than "Finished." While they both mean theoretically the same thing (the quiz is done and submitted), it would differentiate between those submitted by the student (actually clicking the "Submit all and finish" button), and those auto-submitted by the browser (whether due to time running out while the student is in the quiz, or when time ran out with the student not in the quiz; i.e. browser window is closed). This may alert instructors that there was an anomaly with the student's quiz, and they can investigate if so desired. 

This would be contingent, of course, that there are answers to review once the quiz auto-submits.

I'll shut up now and let others reply. gwenu

 

Cyfartaledd sgoriau: -
In reply to Joseph Rézeau

Re: Quiz Answers Removed when Student is not in the Quiz Attempt - and it Auto Submits

gan Jean-Michel Védrine -

Hello Joseph,

The "story" in your Point of view #1 did happen at my institution some years ago, except the student realized her mistake not the next day but just a few minutes after the end of the exam. The decision (please, note I was not involved in that, it was not my exam and not my student winc) of the teacher was to grade her exam normally without any penalty, motives of the teacher for the decision were :

- it was not possible for her to do a lot of things in "such a short time" (?)

- she was "desperate and crying" (??)

- she was a "good student" and surely would not cheat (???)

At that time I was convinced that if she had been a boy the end of the story would have been different gwên fawr

This is also one of the reasons I prefer grading done by computers : no subjective decisions winc

Of course this is totally out of the subject

To answer Tim question I must admit I was rather surprised: I don't use auto-saving a lot (I use quiz for training without any time limit) but I was under the impression that auto saving was similar to saving done when a student navigate from page to page in a quiz,  I was wrong.

My +1 to submit auto-saved responses if the quiz is set to automatically submit at the end.

Cyfartaledd sgoriau: -
In reply to Tim Hunt

Re: Quiz Answers Removed when Student is not in the Quiz Attempt - and it Auto Submits

gan Douglas Broad -

Perhaps this particular instance  (closing the browser just before time expires) deserves special attention.  I imagine that this should be a rare instance.  These unsaved/autosaved questions could be granted probationary status  such that the teacher could accept/not-accept on an individual basis based perhaps on a student interview --- What happened? Why?

At least with paper tests, there is a decision making process possible.  Once the electronic records evaporate, there are no options - other than the extremes (award zero credit or retake - which is probably unjustifiable).

Otherwise accepting autosaved answers when the quiz is set to autosave is better than having the answers disappear.

Cyfartaledd sgoriau: -
In reply to Douglas Broad

Re: Quiz Answers Removed when Student is not in the Quiz Attempt - and it Auto Submits

gan Tim Hunt -
Llun o Core developers Llun o Documentation writers Llun o Particularly helpful Moodlers Llun o Peer reviewers Llun o Plugin developers

I just had a look into the code, and discussed this with my colleague Phil. We reached the consensus (which seems to be shared here) that the current behaviour should be considered a bug. If responses have been auto-saved, then they should be used when we automatically submit.

So: MDL-43733.

Should not be too hard to fix when I get the time, but this sort of thing needs to be done carefully.

Cyfartaledd sgoriau: -
In reply to Tim Hunt

Re: Quiz Answers Removed when Student is not in the Quiz Attempt - and it Auto Submits

gan Drew Smith -

Thanks so much, Tim. Most excellent!

Cyfartaledd sgoriau: -
In reply to Drew Smith

Re: Quiz Answers Removed when Student is not in the Quiz Attempt - and it Auto Submits

gan Tim Hunt -
Llun o Core developers Llun o Documentation writers Llun o Particularly helpful Moodlers Llun o Peer reviewers Llun o Plugin developers

What is (I hope) even more excellent is that I now think I have developed a fix. If anyone would like to review or test the code before it is integrated, that would be lovely.

Cyfartaledd sgoriau:Useful (2)
In reply to Tim Hunt

Re: Quiz Answers Removed when Student is not in the Quiz Attempt - and it Auto Submits

gan Drew Smith -

Tim (and all) - 

We have implemented the code here, and have tested it. So far, everything seems to be functioning as designed. Excellent work!

Thank you so much, Tim, for the quick response!

Cyfartaledd sgoriau: -
In reply to Drew Smith

Re: Quiz Answers Removed when Student is not in the Quiz Attempt - and it Auto Submits

gan Henrieta Mikleova -

Hi,

 

I have just tested MDL-43733 but it does not seems to be working. I followed the testing instructions as discribed, but the students answer is not autosaved. When I review the test as the student, it shows that the attempt is finished but not answered.

Cyfartaledd sgoriau: -
In reply to Henrieta Mikleova

Re: Quiz Answers Removed when Student is not in the Quiz Attempt - and it Auto Submits

gan Jean-Michel Védrine -

Hello Henrieta,

With what version of Moodle have you tested it ?

Are you sure your cron is running (and have you waited long enough for it to run) ?

Cyfartaledd sgoriau: -
In reply to Jean-Michel Védrine

Re: Quiz Answers Removed when Student is not in the Quiz Attempt - and it Auto Submits

gan Henrieta Mikleova -

Hey,

thank you for your answer.

I am testing it in Moodle 2.6.2. The cron is running and I have waited more then 30 minutes.

Cyfartaledd sgoriau: -