Gapfill questions not being graded correctly sometimes

Re: Gapfill questions not being graded correctly sometimes

by S. Ali -
Number of replies: 7

Hello Marcus,

 

Is there any update on these issues?

Issues (1) and (3) have again happened in this week's Quiz.

Any assistance will be greatly appreciated.

It may be best if in the interim I refrain from this question types until the issues are resolved.

Looking forward to a resolution to the problems.

 

Regards,

s.ali

 

 

 

In reply to S. Ali

Re: Gapfill questions not being graded correctly sometimes

by Marcus Green -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers

I have only been able to partly reproduce issue 2, the displacement of draggable items when there is more. I say partly because I don't know which browser was being used, though I suspect it is not browser dependent. I am moving towards releasing version 1.7 and I will also be testing against that to reproduce the circumstances.  Version 1.7 will have a different javascript setup so there is a slightly different basis to the drag and drop functionality.

I have not been able to reproduce incorrect marking. I intend to reproduce the screen where you have a reported incorrect answer, but it would be helpful if you can reproduce with a key, i.e. a western letter appended to each of the arabic strings. If not I will attempt to "do it by eye". My testing will be based on setting regular expressions off and setting the question type to insensitive (certainly at first). 

Could you supply an xml dump of the singular/plural question where you have provided the screen dump.

I have not been able to reproduce item 1 and I have never seen that type of error message in my testing. It is however consistent with a Moodle installation where debugging is set to higher than it should be on a production system. Clearly I would rather no error messages are generated whatever the debug setting, but as I said I have not been able to reproduce the error even with debug set to full. I will continue to test.





In reply to Marcus Green

Re: Gapfill questions not being graded correctly sometimes

by S. Ali -

Hello Marcus,

 

I apologize for the delay in responding; it has been a very busy last few days.

The browsers used for testing are Firefox (38.0.1) and Chrome (43.0).

I look forward to testing the version 1.7 when it's ready.

 

Please find attached the xml dump of the singular/plural question with letters as requested.

 

Just tested item 1 again (from Chrome) and obtained the following error:

Notice: Undefined index: p1 in <moodle_home>/question/type/gapfill/question.php on line 244

 The debugging is set to Normal.

However, on the production system it is now set to Minimal until these issues are resolved.

Please note that as mentioned previously, this occurs with Moodle 2.8.5+. The problem does not seem to occur when tested on Moodle 2.9.

 

Regards,

sali

 

In reply to S. Ali

Re: Gapfill questions not being graded correctly sometimes

by Marcus Green -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers

I have just been testing with  Moodle  2.8.2+

Firefox 38.1 and  Chrome 43.0.2357.65 m and MS IE 11.0.96

I cannot get the question to grade incorrectly. I started with the defaults saved in the XML you supplied 

And I have tried every permutation I can think of that might cause it not to grade correctly (regex, case sensitivity etc)

I have created a single page quiz with about 11 questions and tested the drag and drop capability including that question of yours and it worked OK without the undefined index error.

I find it hard to imagine that the Moodle version change is the cause of the problem, but I will bump up to the same version of you 

and rerun the tests. The only issue I have come across is a slight oddity with drag and drop when using MS IE 11.


I'll let you know when I have done some more tests.



Attachment plural.png
In reply to Marcus Green

Re: Gapfill questions not being graded correctly sometimes

by Marcus Green -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers

I have just done some additional testing on Moodle 2.8.6 and against Moodle 2.8.1 under Linux. I have may have a fix for the drag drop oddity under MS IE, but no reproduction of your other experiences so far.

In reply to Marcus Green

Re: Gapfill questions not being graded correctly sometimes

by S. Ali -

Hello Marcus,

 

Sorry for the long delay in responding. I was unable to replicate the grading error on the PC so I contacted some of the students who encountered the problem and asked a few questions.

From their feedback, they all used iOS devices (Mac mini, iPad, iPhone and iMac) and when I tested on an iPad I was able to replicate the error on the first try.

Using the same question above I entered the correct answer for 2 of the words, left one blank and entered a wrong answer for the 4th. Then I answered some of the other questions and went back to that one and changed some answers so that 3 should have been correct and 1 wrong. The result is shown below:

A, B and C should have been correct. Additionally, the sequence displayed for the saved answers are mixed up for the last 2 words but perhaps that may be due to the language being rtl?

I hope this helps in some way.

Best regards,

sali

In reply to S. Ali

Re: Gapfill questions not being graded correctly sometimes

by Marcus Green -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers

Many thanks for the feedback. Any issue with marking then it will be on the server and not the client. I have been intensely testing version of 1.7 of the Gapfill question type on every version of Moodle from 2.5 to 2.9 and I have included your plural form questioning in my tests but have not been able to reproduce your experience. I will test further  The source of the 1.7 release is available at

https://github.com/marcusgreen/moodle-qtype_gapfill/tree/gapfill_1.7

And I will make an additional post as a general announcement to get other people testing it. 

In reply to Marcus Green

Re: Gapfill questions not being graded correctly sometimes

by S. Ali -

Hello Marcus,

Moodle: 2.8.5+ (Build: 20150414)

 

With some preliminary checking from a Windows PC and an iPad, the grading does not look very good.The questions are Arabic/English including the ones in the sample file and it's much worse than before.

Most answers are graded incorrectly.

I also tested with several other question types and the only ones that were graded incorrectly were the gapfill questions. This was both on the PC and iPad.

I will install on Moodle 2.9 and test over the next few days and report back.

 

Regards,

sali