Should quiz auto-save be logged?

Should quiz auto-save be logged?

by Tim Hunt -
Number of replies: 6
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

For reasons I can only partially remember, certain things in the quiz are not logged. And, having recently had to investigate some issues myself, and also help other people investigate things here, I think it would be better to log all these things. But I thought I would ask in case anyone disagrees.

  1. Quiz autosave - I think originally we did not make this log, because auto saves were not 'real' actions by the user. However, autosave running is good evidence that the student is actively working on the quiz (a save is only done after a response is changed).
  2. Quiz processing responses - currently processattempt.php only logs if it changes the state of the attept (e.g. Submit all and finish). In other cases it does not, on the theory that we are about to redirect the user back to attempt.php, and that will log. However, it will make things clearer if processattempt.php also logs when it saves responses for the user.
  3. While checking out the other two, I also noticed that the user clicking on "Try another question like this one" (when that is enabled) is not logged. That is a significant user action, so definitely should be.

The only real down-side of this logging is that it makes the log table (even) bigger. However, having an accurate record of what happened during quiz attempts is important, so I think it is well worth it - but as I say. I am posting here to see what you all think.

(Tracker issue: MDL-71984)

Average of ratings: Useful (1)
In reply to Tim Hunt

Re: Should quiz auto-save be logged?

by AL Rachels -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers
Based on past experience, I would vote to add these log events. Not having taught any course for six years might make my vote not count for much, though.
Average of ratings: Useful (1)
In reply to Tim Hunt

Re: Should quiz auto-save be logged?

by Rick Jerz -
Picture of Particularly helpful Moodlers Picture of Testers
I support the idea to log autosaves. It seems to me that we somewhat frequently get questions from Moodle.org folks about quiz problems, where students might be claiming that their quiz was not saved. Yes, the log file will grow in size, but it's already big, and I believe that the logs should be able to let us know if the system or the user saved their quiz.  (In my Moodle, quizzes represent the most student activity, so there are a lot of log entries for quizzes.)

As I recall, when a quiz has a time limit and that time limit is reached, and the setting is "open attempts are submitted automatically," the current log entry is that the user submitted the quiz. When you mention logging autosaves, I assume that when open attempts are submitted automatically, that the log entry will show that the system submitted the quiz. Right?

Thanks for trying to take care of this issue, Tim.
Average of ratings: Useful (3)
In reply to Rick Jerz

Re: Should quiz auto-save be logged?

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

I want to classify the issue of what data is included in the 'attempt_submitted' log message as a separate question, to be left for later. (Because I can only juggle so many balls at once.) But some intial thoughts about what we should do, when we can:

For attempt submitted, there are acually 3 cases:

  1. Student clicks 'Submit all and finish'.
  2. Student is actively working on the quiz when the timer hits zero in their browser, and submits the quiz. (Or left the quiz open in their browser and wantered off!)
  3. Student is offline when time expires, and the attempt is closed by the scheduled task.

3. is already easy to distinguish. That gets logged as done by 'Admin user', and there are probably some other things (e.g. it gets log as caused by cli, not web or ws.)

Currently, 1. and 2. are not distinguised. As far as the Moodle server is concernted, they are both triggered by something that happened in the student's browser. However, there is enough information to tell them apart, and include that in the log event. (But, probably the student could hack it to log the wrong thing if they really cared - you can never entirely trust what has come from the client-side.)

Also, whatever we do, we should probably do something consistent for attempt_becameoverdue and attempt_abandoned - acutally, no. I think those ones only ever happen on cron, anyway. I think.

Average of ratings: Useful (1)
In reply to Tim Hunt

Re: Should editing quizzes be logged in detail?

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

I am resurrecting this old thread, because we have now found some more things in the quiz which are not logged, namely all the different things that you can do on the 'Edit quiz' page.

We discovered this when we went to see who hand changed the mark for a partuicular question to 0, and when that had been done. (That is what we tend to do when we discover a particular question is a quiz is faulty after students have started attempting it. So, this was not a witch hunt. We just wanted to check when it had happened, and that's when we found it was not logged.)

From there, we started looking, and found we had opened Pandora's box. We now think that these are the additional log events we should add - but I am just posting here to give you a chance to comment, before we acutally do it:

The edit page itself

  • edit_page_viewed This is the only event that currently exists, and it is logged both when the page is viewed, but also when various other actions are proccessed. We are going to change the triggering, so that in future, 'edit page view' means 'edit page viewed', and nothing else.

Slots

Note that, within the quiz code, the separate 'questions' a better referred to as slots. They correspond to rows in the quiz_slots table.
  • slot_created Add question (all options)
  • slot_deleted Delete a question from quiz
  • slot_moved Move question to another position
  • slot_mark_updated Update max mark for question
  • slot_requireprevious_updated Update dependency of questions (lock icoin)
For the last two here, we could, instead, have a single slot_updated event, and require people to dig into the data to see what is going on, but acutally, I think that two separate events is better.

I think that, for "Delete multiple questions from quiz" it is best to just log this as multiple 'slot_deleted' events, all with the same timestamp.

Pages

Page breaks are not really entities in the quiz, but I still think that these two events are the clearest way to log it.
  • page_break_created Update page break
  • page_break_deleted Update page break
  • section_break_created Add new section heading at page
  • section_break_deleted Delete section heading
  • section_title_updated Update section title
  • section_shuffle_updated Update shuffle option

The whole quiz

  • quiz_repaginated Re-paginate
  • quiz_grade_updated Change the Maximum grade and click on Save
That is a lot of events, but our view is that it is good to have clear logging of all the changes made to quizzes. Also, I should say that most of the credit for producing this list is owed to my colleagues Tien.
Average of ratings: Useful (2)