Quiz

Pierre,
I believe that MDL-8838 grew out of this discussion, so some of your points have already been addressed. Here's my input:

• Category import failure* -- one or more questions do not import
• If I import 100 and 1 fails, should Moodle remove the other 99? Yes
• Should it be an option? Yes, but I wouldn't use it. Default should be "all or nothing"
  

* of course we import categories, not quizzes.

I've been working quite a bit with Moodle's import feature recently, and running into failures everywhere (partly with intent), so here's my two euro-cents from a perspective of an experienced failure...

The most probable reason for an import failure is some kind of format problem. Users get very puzzled about these kinds of thing. What the user most needs to know is where the problem is in their import, so they can weed it out.

With the current "stop-at-point-of-failure" system, the user gets a pretty good clue where the problem is, though the clues could generally be improved.

So if you take away the "stop-at-point-of-failure" system, you need to leave the user with a clue that it is at least as good, if not better, such as "try removing question X and then import again".

Another and perhaps better approach would be maximizing "fault tolerance". During my many import failures, I noticed that Moodle could often correctly count my questions, but stopped long before it had processed them all. This suggests that in these cases, later questions could have been imported if Moodle had wanted to. I'd suggest that higher fault tolerance than at present, coupled with a choice of remedial actions and clear indications of where problems lay, would be user-friendly.

For the inexperienced, losing 100 out of 100 questions could be distressing; getting 90% or so in (and then giving up on the rest) could mean that the test can more or less go ahead on schedule.

(Now, if only I knew why my numerical questions weren't importing properly, but that's another post...)

Cheers,

JM

I like the idea of having an on error delete option. Display the error, let the user fix it and try again. A quiz import failure would be anything that prevents ALL of the questions from being imported - so yes delete the 99 until I get number 100 correct. I have added my vote to the issue.

Pierre Pichet wrote "Howard has put an interesting MDL_8838 improvment proposition:"

How about this...

If a question fails to import (for whatever reason), then do the following:

   1) Stop the import, and advise the User that "Question# 23 in the import file, with Question Name xyz, has failed to import."

  2) Ask the user if they would like to

    (a)  "Skip this question, and continue with import?" or

    (b)  "Cancel the import, but keep Questions that have already been imported?" or

    (c)  "Abort the import completely (do NOT keep Questions that have already been imported)?"

With all these options, this should keep everybody happy!!!

Also, giving as much info about which question caused the problem is also helpful... For example, the specific question number (ie: the 23rd question in the file), and the question name.

This discussion reminds me of early compilers used by programmers in the old days. When they hit the first error, they would stop and say "error on line xxx." You had to fix the error and resubmit the job, not knowing how many other errors there might be. If you were unlucky, the compiler told you there was another error on the very next line (and so on). It was a big improvement when compilers were able to recover and continue diagnosing all the remaining errors. Even though the compilation failed, you got to see all your errors and you could fix them all at once. This was important to us because you could get maybe one or two turnarounds a day back then (I used to get one a day).

I suggest that category import should do the same thing: attempt to diagnose all the errors and then stop without updating the question database unless the file was error free.

The problem with partial imports is how to handle residual questions. Let's use GIFT format as an example (any format will do just as well). Do you cut your GIFT file into pieces in order to import the residual questions? Ugh! Or do you keep your GIFT file intact? Speaking for myself, I discard my GIFT files on philosophical grounds, because to me, the Moodle question database is the true database, and my GIFT files are merely an artifact of loading the database. Once the database is loaded, subsequent changes should be made directly to the database. But I know other people who retain their GIFT files. They regard their GIFT files as the true database because (for example) they're searchable, while the Moodle question database is not. Incidentally, I think a lot of the people who do this don't understand the difference between "quiz" and "category," but that's another discussion. For these people, it is important that the GIFT file be error-free. That's why I think we should NOT allow partial import.

An option "Simulate an import" would answer most of the problems.
The feedback will allow the user to modify his file until it passes all tests.

For less "computer" or "format file" friendly user an additional page listing the acceptable questions with select buttons will be more convenient.

Pierre
Acceptable questions are those that need additional edition like a matching question with less than three options, the moodle standard but not for other systems(as WebCT).

Pierre

There are some practical difficulties to all this (of course)...

• The import formats are pluggable resources. To a large extent the error handling depends on the guile of the particular developer. BUT... this doesn't mean to say that we should not handle the standard formats better.
• The import is effectively a two (well two-and-a-half I suppose) pass process - the file is parsed and the resulting questions are built in an array. The array then gets a bit of pre-processing and is then run through again to call the questiontype methods to write the data to the database. Point being, there are a number of somewhat disconnected points of failure.

I would like to handle errors better in the import/export api. That would be to remove the task of reporting errors from the plugins completely so that it can be handled more consistently. My hope is that this delete everything issue would logically fall out of that. I need to have a bit more of a think about what that might look like.

Screenshots from a first "Simulate import" proposal can be seen on MDL-8838.
Pierre

I think I've spotted something important which nobody else has mentioned yet - but it follows on from the question about what to do with residual questions.

A story of import failure...

Attempt one: import 6 out of 20, with one skipped.

Attempt two: import 13 out of 20, reduplicating the original 6 (aaagh!).

It might be that reduplication which had everyone talkíng about deleting the original 6 after attempt number 1.

But surely (and isn't this the point that nobody brought up yet?) it would be better simply to have a little checkbox saying "don't reduplicate".

OK, I know a function for checking question identity would be a pain to write, but just think of the elegance of the user experience!

Just an idea.

Cheers!

JM