Clone of Quizzes taking hours

Clone of Quizzes taking hours

'mei a Bob Derezinski - 'aho
Number of replies: 14

Hello - we recently updated out Moodle site to version 3.5

Since then we have numerous complaints from our staff stating that when they try and clone a quiz it takes an extremely long time.  This past weekend a user started it on Friday evening and it didn't complete until Sunday.


Is there something wrong with the clone quiz program?


Everything else appears to be working.


Thank you

Average of ratings: Useful (1)
In reply to Bob Derezinski

Re: Clone of Quizzes taking hours

'mei a Emma Richardson - 'aho
Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Plugin developers

How large is the question bank and is it at the system level?

In reply to Emma Richardson

Re: Clone of Quizzes taking hours

'mei a Bob Derezinski - 'aho

It is a very small quiz, only 6 questions.  It is at the system level.

In reply to Bob Derezinski

Re: Clone of Quizzes taking hours

'mei a Emma Richardson - 'aho
Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Plugin developers

If your question bank is at the system level, every time you copy the quiz, it is copying the entire question bank.  The larger the bank, the longer to copy.  However, 2 days seems ridiculous...I used to have a system level quiz bank with a few hundred questions in it, maybe up to 1000.  I ended up moving everything back down to the course level just because of the backup and restore issues.  I was unable to duplicate a quiz at all - it would time out on me every time.

In reply to Emma Richardson

Re: Clone of Quizzes taking hours

'mei a Bob Derezinski - 'aho

Hi Emma - thanks for this info.  One question though.  When you say it copies the entire question bank, is that the entire question bank as it relates to this one quiz, or the entire question bank for all the quizzes in all the other courses as well?  Do you know of a way to delete the old copies?

Thanks again

In reply to Bob Derezinski

Re: Clone of Quizzes taking hours

'mei a Emma Richardson - 'aho
Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Plugin developers

If your question bank is all at the system level, it will copy the ENTIRE question bank.  Tim might correct me here but I believe it copies every part of the bank that the quiz has access to (i.e: all questions from specific course that the quiz is in, all category level and and all system level questions.)

In reply to Emma Richardson

Re: Clone of Quizzes taking hours

'mei a Bob Derezinski - 'aho

Hello - do you know if the questions are at the course category level will it it still copy the entire question bank at that level?

In reply to Bob Derezinski

Re: Clone of Quizzes taking hours

'mei a Rick Jerz - 'aho
Picture of Particularly helpful Moodlers Picture of Testers

I am jumping in a little late, but know what you mean.  Duplicating a quiz does take longer than it should, but it seldom takes my (small) Moodle (with lots of questions) more than 10 minutes.

I am not sure what you are trying to accomplish.  Are you duplicating to reuse the exact quiz somewhere else in your course, or are you duplicating to keep your quiz settings, while wanting to change questions?

What I most often do in a new course is to create a quiz with all of the settings set, but no questions.  Then I duplicate this "shell" quiz, moving them into topics.  Then I go into each quiz, make some title changes, and finally add questions.  I hope that you understand what I am saying.  Continue asking questions, if needed.


Average of ratings: Useful (1)
In reply to Rick Jerz

Re: Clone of Quizzes taking hours

'mei a Richard Lobb - 'aho
Picture of Plugin developers

We just had a problem of this sort, with quiz duplication taking many hours, and it turned out to be due to a recently identified critical moodle bug in versions 3.5 and 3.6: MDL-63260 - "Duplicating quiz with random question has exponential questions growth". Every time you duplicate a quiz in a course that has random questions in it, the number of random questions in that course doubles. So if you start with a single random question and over time you duplicate 20 quizzes, you now have around 1 million stray questions. On our server it turned out we had 4 million questions in one course rather than the expected 2000 and quiz duplication (which, like all activity duplication operations involves a backup-restore cycle) was taking hours. Various other operations (e.g. course backups) suffer drastically as well.

The stray questions aren't visible through the web interface but a system administrator can run a database query to check the total number of random questions, e.g.

SELECT count(id) FROM mdl_question WHERE qtype='random';

A (partial) fix was made available at the end of December 2018. It involves running a background task ("Remove unused random questions") to identify and delete such stray questions. It removes up to 10,000 stray questions each run. See  MDL-63260 for details. So we pulled the latest version of Moodle, ran the task manually once to see how long it took (7 minutes to delete 10,000 questions) and set the task frequency via the administrator scheduled tasks panel to run every 9 minutes. After 12 hours we're down from 4 million stray questions to just under 1 million. This is faster than expected, which is mildly concerning, but otherwise it's all looking good. 

Average of ratings: Useful (4)
In reply to Richard Lobb

Re: Clone of Quizzes taking hours

'mei a James Roberts - 'aho

Richard,

Thanks for this answer. I have noticed this problem recently and I will advise my administrator (we use Moodle 3.6 but which specific update, I'm not sure). In my case, I sometimes receive an Alert/Message box that says simply "undefined".

Note, this problem crashes use of Moodle in the browser. I am forced to reboot both my browser (Chrome) and of course Moodle. The issue is clearly related to the size of the question bank; not the complexity of the quiz being duplicated nor whether the question bank is at system level.

I recently duplicated a quiz activity in a course with no problem. I then added many questions to the same course's question bank (using the sharing cart) and now it is in effect impossible to duplicate a quiz activity in the course.

A workaround for people at the teacher level (ie. other than a SQL query/cleaning at administrator level) is to create a new quiz activity and then manually add questions as required.

Hope this problem is resolved soon. 

In reply to James Roberts

Re: Clone of Quizzes taking hours

'mei a James Roberts - 'aho

Oops. Or as the kids say, "My bad". (Apparently this problem has been fixed in a later version of Moodle 3.6 than what I am using.)

Note to Tim Hunt: Comme d'habitude, we all owe you a tremendous vote of thanks.

In reply to Richard Lobb

Re: Clone of Quizzes taking hours

'mei a Chris Kenniburg - 'aho
Picture of Particularly helpful Moodlers Picture of Plugin developers

Apparently this bug has gotten us REALLY good.  I just noticed on our Moodle Registration page it says we have over 17 MILLION questions in our question bank.  I'd like to think our teachers are using it that much but I know differently!  

We just discovered this and I will be updating our Moodle site today.   Is the only remedy to run the 

php admin/tool/task/cli/schedule_task.php --execute=\\qtype_random\\task\\remove_unused_questions

to eliminate the questions.


Chris

Average of ratings: Useful (1)
In reply to Chris Kenniburg

Re: Clone of Quizzes taking hours

'mei a Richard Lobb - 'aho
Picture of Plugin developers

You don't have to run the task from the command line - it is automatically scheduled to run every hour by the system task scheduler. The exact schedule is set via the administrator Scheduled Tasks page.

When trying to clean our production system I set it to run every 9 minutes, because I estimated that it would take around 8 minutes to delete 10,000 questions before quitting. But I now suspect it automatically reschedules itself if more questions remain at the end of its run. So you may not need to do anything other than upgrading your system and leaving it to sort itself out. However, I haven't checked the code so I'm not sure it reschedules itself; setting it to run every 10 minutes rather than once per hour should be a harmless way of making sure you get a quick-ish fix.

Once it has cleaned the question bank, the problem is solved, so I shouldn't have referred to it earlier as a "partial fix". It is a complete fix. As another poster says: "Thanks for the great effort on this, Tim Hunt".

Richard

In reply to Richard Lobb

Re: Clone of Quizzes taking hours

'mei a Chris Kenniburg - 'aho
Picture of Particularly helpful Moodlers Picture of Plugin developers

Thanks for the follow up and outstanding work of Tim Hunt.  

Yesterday at 4pm we upgraded and saw the new scheduled task.  On our server - judging by the cpu spike -  it looks like the process takes about 2 minutes to complete and it was set to run every 15 minutes or possibly on the hour at 15 minutes.  Well, since school was out we made it run every 2 minutes.  

It went from 17,884,346 questions yesterday down to 13,775,188 questions this morning.  

We switched back to every 5 minutes and are monitoring performance hits as school is in session.  We haven't noticed anything yet.  

On the bright side of all this... We know our servers can handle many, many, many more quiz questions from teachers!!!

The upgrade seems to be working and hopefully by next week we will be back to normal.  I can't believe we didn't catch this question growth sooner but glad there was such a nice fix and cleanup solution.


Is there any reason I shouldn't set the time interval so low?


Thanks again,

Chris

Average of ratings: Useful (3)