1.9 upgrade quiz duplicate entry error

1.9 upgrade quiz duplicate entry error

by Oliver Cooper -
Number of replies: 16

Hi all,

Been ripping my hair out over this one, trying to upgrade our 1.9 install to 2.2. Upgraded to the latest build of 1.9 first. I've also tried going to 2.1 first instead of straight to 2.2. - same problem.

Obviously I'm not doing this on the production server :P Each time it falls over I revert back to its original state.

Most of the upgrade process seems to complete quite happily then it falls over when upgrading quiz attempts.

I've found a few other instances of people having the same problem but finding it hard to follow steps to resolve. I'm quite capable of navigating the database and modifying it where required but I'm not exactly a SQL whiz so not so great at writing queries etc.

The database is MySQL.

Any help is greatly appreciated, need to get this shifted to 2.2 as soon as.

Here's the error:

Error writing to database

Debug info: Duplicate entry '3361-1' for key 'mdl_quesatte_queslo_uix'
INSERT INTO mdl_question_attempts (questionid,variant,behaviour,questionsummary,rightanswer,maxmark,minfraction,flagged,responsesummary,timemodified,questionusageid,slot) VALUES(?,?,?,?,?,?,?,?,?,?,?,?)
[array (
0 => '271',
1 => 1,
2 => 'adaptive',
3 => 'Look at the image below and then match the statements up correctly below {60% of heat is lost through the; 25% of the heat lost is through the; This could be used to prevent heat loss through the roof; This could be used to prevent heat loss through the windows; This process is when the gap between the walls in a house is filled with foam} -> {walls and roof; windows and doors; Loft insulation; Double glazing; Floor and doors; Pipe lagging; Chimney; Cavity wall insulation}',
4 => '60% of heat is lost through the -> walls and roof; 25% of the heat lost is through the -> windows and doors; This could be used to prevent heat loss through the roof -> Loft insulation; This could be used to prevent heat loss through the windows -> Double glazing; This process is when the gap between the walls in a house is filled with foam -> Cavity wall insulation',
5 => '1.0000000',
6 => 0,
7 => 0,
8 => 'This could be used to prevent heat loss through the windows -> Double glazing; This could be used to prevent heat loss through the roof -> Loft insulation; 60% of heat is lost through the -> walls and roof; 25% of the heat lost is through the -> Cavity wall insulation; This process is when the gap between the walls in a house is filled with foam -> Loft insulation',
9 => '1350500844',
10 => '3361',
11 => 1,
)]
Stack trace:
  • line 397 of \lib\dml\moodle_database.php: dml_write_exception thrown
  • line 893 of \lib\dml\mysqli_native_moodle_database.php: call to moodle_database->query_end()
  • line 935 of \lib\dml\mysqli_native_moodle_database.php: call to mysqli_native_moodle_database->insert_record_raw()
  • line 299 of \question\engine\upgrade\upgradelib.php: call to mysqli_native_moodle_database->insert_record()
  • line 253 of \question\engine\upgrade\upgradelib.php: call to question_engine_attempt_upgrader->insert_record()
  • line 221 of \question\engine\upgrade\upgradelib.php: call to question_engine_attempt_upgrader->save_usage()
  • line 176 of \question\engine\upgrade\upgradelib.php: call to question_engine_attempt_upgrader->convert_quiz_attempt()
  • line 131 of \question\engine\upgrade\upgradelib.php: call to question_engine_attempt_upgrader->update_all_attempts_at_quiz()
  • line 1112 of \mod\quiz\db\upgrade.php: call to question_engine_attempt_upgrader->convert_all_quiz_attempts()
  • line 545 of \lib\upgradelib.php: call to xmldb_quiz_upgrade()
  • line 271 of \lib\upgradelib.php: call to upgrade_plugins_modules()
  • line 1461 of \lib\upgradelib.php: call to upgrade_plugins()
  • line 317 of \admin\index.php: call to upgrade_noncore()
Average of ratings: -
In reply to Oliver Cooper

Re: 1.9 upgrade quiz duplicate entry error

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

So, as part of the upgrade, we are generating some data that violates a unique index in the database.

This suggests that the data from before the upgrade had problems, but that bad data did not cause problems before.

So the problem is to try to track down the problem in the data before you start the upgrade. Once we know what the problem is, we may be able to fix it manually.

Anyway, you need to restore the 1.9 data again, and then we need to run some queries on that.

I think the first thing to check is that your 1.9 database has a unique index on the quiz_attempts.uniqueid column.

In reply to Tim Hunt

Re: 1.9 upgrade quiz duplicate entry error

by Oliver Cooper -

I've located the uniqueid column/mdl_quiz_attempts, what exactly am I looking for/need to run there?

In reply to Oliver Cooper

Re: 1.9 upgrade quiz duplicate entry error

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

You need to do

SHOW INDEXES FROM mdl_quiz_attempts

and see if there is an unique index on the uniqueid column.

In reply to Tim Hunt

Re: 1.9 upgrade quiz duplicate entry error

by Oliver Cooper -
Table Non_unique Key_name Seq_in_index Column_name Collation Cardinality Sub_part Packed   Index_type  
mdl_quiz_attempts 0 PRIMARY 1 id A 4119 NULL NULL   BTREE  
mdl_quiz_attempts 0 mdl_quizatte_uni_uix 1 uniqueid A 4119 NULL NULL   BTREE  
mdl_quiz_attempts 1 mdl_quizatte_use_ix 1 userid A 4119 NULL NULL   BTREE  
mdl_quiz_attempts 1 mdl_quizatte_qui_ix 1 quiz A 100 NULL NULL   BTREE

This is what it says, any thoughts? :P

In reply to Oliver Cooper

Re: 1.9 upgrade quiz duplicate entry error

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

OK, so the data must be unique before the upgrade starts.

We still need to try to identify what is causing this error. Ah! but we know. It is the attempt with uniqueid 3361, so please could you run this query: http://docs.moodle.org/dev/Question_database_structure#Following_what_happens_when_questions_are_attempted on the un-upgraded database. Edit it to use the WHERE clause

WHERE quiza.uniqueid = 3361

Then attatch the data here. (Sorry it took me so long to realise that was the thing to do.)

In reply to Tim Hunt

Re: 1.9 upgrade quiz duplicate entry error

by Oliver Cooper -

query won't run, tells me "#1064 - You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'varchar) END AS event, st.answer, st.raw_grade, st.grade, st.pen' at line 10"

Sorry if I'm being a bit of an idiot :/

I should also point out that on other ocassions, the installer has stopped with the same error but moaned about different entries so it may not just be this one uniqueid.

In reply to Oliver Cooper

Re: 1.9 upgrade quiz duplicate entry error

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

It is buried in the paragraph of text after the query:

does not quite work in MySQL. You need to change 'CAST(st.event AS varchar)' to 'st.event' in the middle

We just need to look at the data from one or two problem attempt ids.

In reply to Tim Hunt

Re: 1.9 upgrade quiz duplicate entry error

by Oliver Cooper -

Output for the query is attached

In reply to Oliver Cooper

Re: 1.9 upgrade quiz duplicate entry error

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

I am sitting here scratching my head. That looks like the data for a completely ordinary quiz attempt (with a matching questions and some numerical and multiple choice questions, in adaptive mode). I am sure millions of attempts like that have been successfully upgraded.

Why else could it be failing?

What is suppose to happen during the upgrade is this:

  1. We rename the old question_attempts table -> question_usages, and change the definition a bit.
  2. We create new tables called question_attempts (yes, the same name as before) question_attempt_steps and question_attempt_step_data.
  3. Then we fill those tables with data that comes from converting the old data in question_sessions and question_states.

And, during step 3, you get a unique key violation.

Ah, now, in step 2, when we create the new tables, actually, if any of the new tables already exist, we don't create them, and so they could be there and contain data.

Except that, because of the rename in step 1, that cannot be happening with the question_attempts table.

So, all the data going in to that table must be generated during the upgrade.

The code that does step 3 is in question/engine/upgrade/upgradelib.php. The flow of code is convert_all_quiz_attempts -> update_all_attempts_at_quiz -> convert_quiz_attempt -> save_usage.

I think the next step of debugging, if you are up to it, is to add lots l statements like:

print_object("Starting conversion of quiz " . $quiz->id);

print_object("Starting conversion of attempt " . $attempt->id);

all over the place, so you can see exactly what it is doing. Then run the upgrade again, and hopefully then when it fails, we will understand what is happening.

Average of ratings: Useful (1)
In reply to Tim Hunt

Re: 1.9 upgrade quiz duplicate entry error

by Oliver Cooper -

You don't think it could be anything to do with the browser that's causing it possibly? - i leave the install going overnight or during the day, i get back to it and it's stopped doing anything, normally the browser appears to be constantly loading as the progress is displayed or at least the dots keep piling up while working through the quiz attempts. Once it stops doing anything that i can see there is no error, just sits looking normaly but doing nothing.. the error is displayed when i refresh the page.

In reply to Oliver Cooper

Re: 1.9 upgrade quiz duplicate entry error

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

That is almost certainly the problem.

If at all possible, you should be doing the upgrade via the command-line: http://docs.moodle.org/23/en/Administration_via_command_line

In reply to Tim Hunt

Re: 1.9 upgrade quiz duplicate entry error

by Oliver Cooper -

Windows install I'm afraid sad wondering if it'll be worth trying shifting it onto Linux to attempt the upgrade then back to win. Unless there's any advice you can give me for making the upgrade work on win.

In reply to Oliver Cooper

Re: 1.9 upgrade quiz duplicate entry error

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

That docs page is written assuming that you are using Linux. However, the CLI scripts may run on Windows too. I must admit I have never tried it, but there is no real reason it won't work.

In reply to Tim Hunt

Re: 1.9 upgrade quiz duplicate entry error

by Oliver Cooper -

looks like I can't run it on Windows :/ tried running the upgrade in Firefox instead of IE and same problem, installation appeared to stop responding, the moodle site was accessible again... refresh the stopped install page and it has the same error (different key everytime).

Debug info: Duplicate entry '1666-1' for key 'mdl_quesatte_queslo_uix'

In reply to Oliver Cooper

Re: 1.9 upgrade quiz duplicate entry error

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

A thought: are you using MyISAM or InnoDB tables?

I would expect, if you are using InnoDB, then when it times-out, the stuff that is currently in progress will get rolled back, and when you reload, it will continue from where it had got to.

If you are using MyISAM, then all bets are off. MyISAM does not support transactions.

http://docs.moodle.org/19/en/Migration_from_MyISAM_to_InnoDB

In reply to Tim Hunt

Re: 1.9 upgrade quiz duplicate entry error

by Oliver Cooper -

Apologies for the delayed response, had a hectic week at work.

Moved it over to my macbook and tried running the upgrade and it worked! Must be a PHP or SQL timeout on the server, can fiddle with that before doing the proper upgrade now.

You've been a star, thanks so much for the help! smile