Questions already answered; student seeing what he should not see

Questions already answered; student seeing what he should not see

by Ken Masters -
Number of replies: 6

Ver 3.1

I ran a quiz for 192 students.  The quiz was set up with questions in groups of 5, and in each group, only the first question was visible, with each next question becoming visible only when the previous question was answered. (I did this by using the Layout / Navigation Method / Free and Question Behaviour / How Questions Behave / Immediate Feedback and then using the lock).

For all students, this worked fine, except:

Problem 1:

Student A calls an invigilator with two problems:

  1. In the first part of the quiz, many of his questions have already been answered, but not by him.
  2. The later portion of the quiz has all the questions visible to him, even though he has not yet answered the qualifying questions.

We log him out of Moodle and then back in.  Problem still exists.

We then move him to another machine, he logs in, and, again, the problem still persists.

So, I quickly set up one of the invigilators as a student in my course, she logs in, and the student completes the quiz, no problem.

Problem 2:

When I come to check the marks in the exported spreadsheet, I know to copy the marks from the invigilator's row to his row.  Problem is, he is listed in the spreadsheet, with a full row of marks, even for those questions that I know he did not answer.  The logs show him as having logged in at the correct time of the quiz, and answering all the questions over 1.5 hours, even though we know he was in the quiz for a max of 20 minutes.

So, how did he get those marks?

Problem 3:

Student B whom I know was at the exam, has a blank row of marks.  Nothing at all.  I check the logs, and there is no record that Student B was at the exam.   Given the time shown on the Student A marks, it is likely that the marks showing for Student A come from Student B.

Now, even if they had somehow colluded, it would not explain how Student A was able to see all the questions even though he had not answered the qualifying questions.


So, I desperately need to find out what has happened, because the implications obviously go way beyond this one quiz and these two students.  (And right now, can I be absolutely certain that those marks recorded for Student A actually do come from Student B?)

I have not yet had a chance to raise this with our technical staff, so it would be useful if I had some insight beforehand to ask them to look for possible solutions and preventative action.

Any ideas would be helpful.

Thanks.






Average of ratings: -
In reply to Ken Masters

Re: Questions already answered; student seeing what he should not see

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

I think the most likely scenario is:

  1. Student A sits down at one computer and logs in (perhpas even earlier in the day)
  2. Student A then leaves without logging out.

  3. Student B goes in to the exam, and sits at the computer where Student A was working earlier in the day. Starts the exam without noticing that they are logged in as someone else.

  4. Student A goes in to start the exam, logs in at a different computer, and starts the exam, can see the answers that student B has already stared entering.
Probably the way to try to work this out is to look in the server logs and Moodle logs, and filter them by the IP addresses of the two computers involved.
Average of ratings: -
In reply to Tim Hunt

Re: Questions already answered; student seeing what he should not see

by Ken Masters -

Hi Tim

  1. Thanks very much for this.  Since your post, I have had a look at Student A's logs, and yes, it appears to show him accessing the quiz at the same time from two different machines.  I will go over it in detail with our tech staff, but at a first look, it seems as if he was logged into Moodle shortly before the exam, and then switched machines just before taking the exam.  We have a 10-minute inactivity logout, but it would mean that he forgot to logout out of Moodle, and then the second student arrived within 10 minutes, saw Moodle already logged in, and did not log out and then accessed the exam.  So, it would require both of them to have been silly.  Not impossible, I guess sad.
  2. I'll have to get hold of both students and chat to them, but this does seem like a likely cause. 
  3. While this does make me feel a bit better, it still does not explain why Student A was then still able to see the questions in the later part of the quiz without having to answer the qualifying questions.  I should really set up test accounts and hammer this and try to re-create the problem.  (Unfortunately, we are denied having test accounts, and I cannot ask my colleagues to let me have their login passwords so that I can test this, so I'll have to just hope it does not happen again, and alter the exam instructions to include information about checking that the name displayed at the top of the screen is theirs, and have the invigilators double-check also.)

Thanks very much.


Average of ratings: -
In reply to Ken Masters

Re: Questions already answered; student seeing what he should not see

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

Each user can only have one active quiz attempt. So, if you log in as the same user on two different computers, and go to a quiz, they will both be looking at the same attemtp, which will show the same thing to both people (although if they both try to change the same question at the same time, one of them will get an error.)

You can easily reproduce this situation yourself, to see exactly what happens.

Average of ratings: -
In reply to Tim Hunt

Re: Questions already answered; student seeing what he should not see

by Ken Masters -
Hi Tim


Yes, thanks, that part you explained clearly, and your explanation helped us identify why the one student saw some of the questions already answered.

The second problem, however, was that the quiz questions had been set up in groups of 5, and, in each group, questions could only be visible when each preceding (qualifying) question had been answered.  In this case, the qualifying question had not been answered (they were in the second part of the quiz and neither student had reached there yet), and yet the student could see the questions before they were supposed to be visible.  

I don't know if the two errors were related (i.e. the double-taking of the quiz somehow causing the other questions to be seen), or if the two errors are unrelated, and simply coincidentally happened to the same student.

Regards



Average of ratings: -
In reply to Ken Masters

Re: Questions already answered; student seeing what he should not see

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

I think we need more information to work it out. ... Reads back to start of thread:

OK, so immediate feedback. (What question type?)

Are you sure that this only happened to this one student?

Looking at the relevant code (https://github.com/moodle/moodle/blob/master/mod/quiz/attemptlib.php#L1158) there is nothing that would behave different depending on who the student is.

Average of ratings: -
In reply to Tim Hunt

Re: Questions already answered; student seeing what he should not see

by Ken Masters -

Hi Tim

The questions were all MCQs, one correct answer.

Well, it was reported by only one student, and I probably would have noticed if it it had happened to a larger number. It's quite possibly tied to the dual login.

It the meantime, I have requested our tech support to have the student number displayed at the top of the quiz page, and then invigilators can match the student number on the student's ID card with the number on the screen, to reduce the chances of this happening again by accident.

Thanks

Regards




Average of ratings: -