JSME qutype bug ?

JSME qutype bug ?

by Ian Hunt -
Number of replies: 19

We are currently running Moodle 2.5.1+ (Build: 20130823).  We make extensive use of the JSME qu type in organic chem courses and all has been great until 2016 started.  In the last week, we have observed a bug that is prohibitive but we can't isolate the cause.

In some practice materials we allow "check" of answers to promote learning.  Intermittently but regularly, we observed that the current drawing is being ignored and the answer being returned corresponds to the previous drawing (hence most obvious when it's the first drawing and the previous is a blank canvas).  I have seen the problem on students work, but I can not reproduce it in my role. However, if I switch to a student role, then I can replicate the problem. I have tried different browsers and different computers both inside and outside the computer lab and the bug is reproducible...

Any ideas?  

Average of ratings: -
In reply to Ian Hunt

Re: JSME qutype bug ?

by Marcus Green -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers

What version of the question are you running (I know nothing about the question type, but it might help someone who does)

Average of ratings: -
In reply to Ian Hunt

Re: JSME qutype bug ?

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

We use this quetsion type at the OU (with Moodle 2.9 at the moment) and I have not heard any reports of problems like this.

Average of ratings: -
In reply to Tim Hunt

Re: JSME qutype bug ?

by Ian Hunt -

We looked at upgrading to a newer versions of Moodle in the summer but encountered some problems that we could not resolve in time for the start of the fall semester. Winter semester has now started, so any upgrade is not going to happen until May.  We had not had problems until this week.... and we are having a hard time working out what might have changed to create the bug.  Any chance I can get a guest log in to the OU courses to see if I can observe the bug ?

Average of ratings: -
In reply to Ian Hunt

Re: JSME qutype bug ?

by Marcus Green -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers

What version of the question type are you running?, You should be able to see this information at  yourmoodle/admin/qtypes.php

What problems are you seeing?

Average of ratings: -
In reply to Marcus Green

Re: JSME qutype bug ?

by Ian Hunt -

Just back from class and got a reply to this question, I'm told it's 2014080800. 

 I described the bug in a previous post.  It only appears for students, i.e. I can only replicate the bug when in student mode. It is most obvious when we are in practice materials where the students have the ability to check the answer before moving on. On occasion, the drawing tool returns the previously drawn answer and not the answer just submitted. It is most obvious when the previous answer was blank.  It does not happen every time, but maybe 1 on 20, but then might happen repeatedly for 4 or 5 times in a row.  The concern we have is that if we are running assignments for grades with a 1000 students with 20 questions, this is too unreliable....

Average of ratings: -
In reply to Ian Hunt

Re: JSME qutype bug ?

by Ian Hunt -

On our Moodle 2.5 install with the JSME qu type (version 2014080800) we have encountered a bug where the question sometimes fails to return the most recent drawn structure and instead returns the previous answer.  We found this reported at :

https://github.com/jmvedrine/moodle-qtype_jme/commit/6c1fb2e8b6e358a9d2dd647b6be0feef098318b3

Where other than the title, there is no further description. In our hands we can recreate the bug in various browsers and PC or Mac. However, it seemed that I had to be in a student role to produce the bug.  When I test for the bug, I work a question in interactive mode and keep drawing / checking to see in the bug appears.  This seems to be the best way to replicate the effect of hundreds of students working say 10 JSME questions. Replicating the bug in a single submission assignment is less likely to reveal it's presence.

So we felt that we needed to update Moodle and set up a Moodle 2.9 test site today with JSME version 2015051100.  Hoping we had solved our problem I tested it out but found I was able to recreate the bug.....  :  (

While others have mentioned that they haven't encountered the bug, given that we still see it, I wonder whether it is actually there just undetected so far.... I am going to check a 3rd party installation over the weekend to see if I can reproduce it for their install.

Any insight would be appreciated !


Average of ratings: -
In reply to Ian Hunt

Re: JSME qutype bug ?

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

Ah, I have just spotted that you are using qtype_jme. The question type we are using is qtype_pmatchjme. I don't know if there are any significant differences in how the Moodle qtype code interacts with the JSME widget that might affect this.

Average of ratings: -
In reply to Tim Hunt

Re: JSME qutype bug ?

by Jean-Michel Védrine -

The JME question type latest version works exactly as the pmatchjme question type for the way the editor is integrated and how answers are transmitted from the editor to the question.

I have tried to replicate the problem but unfortunately I have been unable to see it, most probably because I have very limited time available.

Now the sad news

I will stop working on a lot of question types the I have maintained until now. I am sorry but I need to do that and I think it's honest to announce it here.

I have not quite decided what question types I will continue to maintain but most probably I will only work on qtype_formulas because I have spend so many time on this one that I can't leave it like that.

So that means that:

  • qtype_algebra
  • qtype_ddmatch
  • qtype_jme
  • qtype_multichoiceset
now need a new maintainer ...

For the other plugins, I will try to maintain mod_qcreate if I can, because I must admit I like it.
But even for the plugins that I will maintain, most probably the support will be slower and, as I will not use them anymore with my students, it will most likely be limited to fix bugs when they are reported.
I discovered Moodle many years ago when it was at version 1.2 or 1.3 I think, then latter I began contributing to this forum and to core code and plugins.
It was a lot of fun being a member of this community and I definitely think that Moodle has made my teaching better and helped my students.
I wish Moodle and all Moodlers a happy future.
Average of ratings: -
In reply to Jean-Michel Védrine

Re: JSME qutype bug ?

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

Thank you very much, Jean-Michel, for everything you have contributed to Moodle over the years. It personally makes me very sad that you are having to scale back your involvement because it has been a great pleasure collaborating with you. However, I understand that you have given a lot, and you are entitled to stop at any time.

I hope you too have a very happy future.

Just to let you know that I have messaged David Mudrak, who looks after the plugins DB, with a link to your message so that he can put most of your plugins up for adoption.

Average of ratings: -
In reply to Jean-Michel Védrine

Re: JSME qutype bug ?

by Marcus Green -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers

Thanks for your huge contributions Jean-Michele, not only for your excellent plugins code but also for the support you have given to others such as myself both in the forums and directly. I just pulled this comment from some of my own code as an illustration 

/* Thanks to Jean-Michel Vedrine for pointing out the need for this and delete_files function */


Average of ratings: -
In reply to Marcus Green

Re: JSME qutype bug ?

by Mary Cooch -
Picture of Documentation writers Picture of Moodle HQ Picture of Particularly helpful Moodlers Picture of Testers Picture of Translators

I too am very sad that you feel unable to continue your level of involvement because it has been of great help to many in the community. I particularly appreciate your efforts with  improving the Lesson activity. I wish you well in the future and hope we do still see you here from time to time. Bon courage smile

Average of ratings: -
In reply to Jean-Michel Védrine

Re: JSME qutype bug ?

by David Mudrák -
Picture of Core developers Picture of Documentation writers Picture of Moodle HQ Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers Picture of Plugins guardians Picture of Testers Picture of Translators
I am sorry to hear that Jean-Michel. Thanks a lot for the time you've put to all the contributions to Moodle and these plugins.
Average of ratings: -
In reply to Ian Hunt

JSME qutype bug is broader in scope

by Ian Hunt -

We continue to fight the bug we encountered in JSME qtype... we have now observed it on another Moodle site (2.8.9+) at another institution and based on that decided to explore the short answer match questions themselves (since JSME is very closely related to that type). 

Today, I was able to replicate the bug in the short answer question type.  I set up a short answer question in adaptive mode and just worked through the letters of the alphabet one at a time. The bug is where a current answer is replaced with the previous answer, most commonly (maybe uniquely) when the previous answer was a blank submission.  Interestingly, submitting a blank and then trying to submit an answer seems to be closest I've gotten to finding a definite trigger.  With both the drawing tool and the short answer questions, it appears that the bug appears most regularly (maybe only) when there is a single keyboard entry or single mouse click with the drawing tool.   We've been using the adaptive mode to provide a way to simulate having hundreds of students submitting answers. 

Average of ratings: -
In reply to Ian Hunt

Re: JSME qutype bug is broader in scope

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

Can you reproduce this on demo.moodle.net, or just on your own server.

Please try to get exact steps to reproduce.

Average of ratings: -
In reply to Tim Hunt

Re: JSME qutype bug is broader in scope

by Ian Hunt -

Yes I can recreate the bug on demo.moodle.net

I created a short answer match question and set up the quiz in adaptive mode, then did a preview.  I entered "a" and got the expected incorrect feedback, I then deleted the "a" (using delete key) and then hit check.  You get the invalid response message. I then entered "a" again and check and the "a" disappears. It did this on the first instance, but I am not 100% sure that that is always the case as it had felt that the bug appeared randomly but in this case it does appear that the delete of a previous answer can be a significant trigger to the bug.  I have used the adaptive mode as the best way to simulate hundreds of students submitting answers. Hope that helps...

I suspect in the JSME qtype, the delete operation equates to the first button in the JME tool, the blank page.

Average of ratings: -
In reply to Ian Hunt

Re: JSME qutype bug is broader in scope

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

Yes. Very clear bug.

  1. Create a short-answer question.
  2. Click the icon to preview it.
  3. Set How questions behave: Adaptive, and click Start again with these options.
  4. Enter 'frog' and click Check. (Incorrect)
  5. Delete 'frog' and click Check. (Incomplete answer)
  6. Enter 'frog' and click Check.

Expected result: The word 'frog' is still visible in the input box, and the state is Incorrect.

Actual result: The input box is blank, and the state is Incomplete answer.

(If you look at the Response history, you will see that a 4th step has not been added, when it should have been.)

Filed as MDL-53011. Thank you for finding this bug. Should be easy to fix now, if only I can get some time.

Average of ratings: -
In reply to Tim Hunt

Re: JSME qutype bug is broader in scope

by Jean-Michel Védrine -

This is good news.

Once MDL-53011 is fixed it should be easy to fix JME (and maybe other question derivated from qtype_shortanswer).

I will try to follow this thread and if I can I will make a new release of qtype_jme, when the problem is solved, but I fear this will be the last release from me.

Of course I will surely visit this forum from time to time in the future.

Average of ratings: -
In reply to Jean-Michel Védrine

Re: JSME qutype bug is broader in scope

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

I think the MDL-53011 bug is entirely within the adaptive behaviour. So, the good news is that once it is fixed, all question types will just start to work.

Average of ratings: -
In reply to Tim Hunt

Re: JSME qutype bug is broader in scope

by Ian Hunt -

I am not sure that it is restricted to adaptive mode.  I just went to a JSME type question (which is closely related to short answer) in an assignment on our Moodle system and I replicated the bug there too.  I had used the adaptive mode to simulate lots of student answers, however as I worked at replicating the bug it became apparent to me that the delete of a previous answer seemed to be a trigger.

Average of ratings: -