General developer forum

Restore error: duplicate entry in mdl_forum_subscriptions

 
Picture of David Vogel
Restore error: duplicate entry in mdl_forum_subscriptions
 

I have backed up a course and am trying to restore it within the same site as a new course. When I do so, I get the following error: 

Default exception handler: Error writing to database Debug: Duplicate entry '6-109' for key 'mdl_forusubs_usefor_uix'
INSERT INTO mdl_forum_subscriptions (userid,forum) VALUES(?,?)
[array (
  0 => '6',
  1 => 109,
)]
Error code: dmlwriteexception

For the course I'm trying to restore, the duplicate entry is always 6, but the other value changes. It's been 109, 127, 129, etc. Looking at the mdl_forum_subscriptions table, the 6 is the user id and the second number is the forum to which the user is subscribed. When I tried to backup and restore another course, I got an identical error message, but for user 7. 

I've looked in the mdl_forum_subscriptions table for the Moodle installation from which I backed up the course and to which I'm trying to restore it (same site), and there isn't a single row with user 6 or 7 and any of the forum IDs which are coming back as duplicates, let alone two duplicate entries. 

Based on other forum threads, I have increased max_allowed_packet to 100M in my.cnf, which did not help. The entire backup file is just ~27M. The database is already in Barracuda format. 

If I restore the course without user data, it works fine. However, that means I lose all the forum posts, which are an important part of the course setup, so that's not an option. However, I don't need the forum subscriptions, if there was some way to just strip that part of the backup file? I've unzipped the .mbz file and dug around, but I don't know what I'm looking for. (And besides, as I said above, the table for the original course doesn't include the subscriptions which are showing up as duplicate in the debugging message.) 

We recently upgraded to Moodle 3.4. The change to Barracuda was also fairly recent, but everything seems to be working apart from this issue. Any suggestions? 


Here is the stack trace

line 489 of /lib/dml/moodle_database.php: dml_write_exception thrown
line 1300 of /lib/dml/mysqli_native_moodle_database.php: call to moodle_database->query_end()
line 1346 of /lib/dml/mysqli_native_moodle_database.php: call to mysqli_native_moodle_database->insert_record_raw()
line 171 of /mod/forum/backup/moodle2/restore_forum_stepslib.php: call to mysqli_native_moodle_database->insert_record()
line 137 of /backup/util/plan/restore_structure_step.class.php: call to restore_forum_activity_structure_step->process_forum_subscription()
line 112 of /backup/util/helper/restore_structure_parser_processor.class.php: call to restore_structure_step->process()
line 178 of /backup/util/xml/parser/processors/grouped_parser_processor.class.php: call to restore_structure_parser_processor->dispatch_chunk()
line 100 of /backup/util/helper/restore_structure_parser_processor.class.php: call to grouped_parser_processor->postprocess_chunk()
line 148 of /backup/util/xml/parser/processors/simplified_parser_processor.class.php: call to restore_structure_parser_processor->postprocess_chunk()
line 92 of /backup/util/xml/parser/processors/progressive_parser_processor.class.php: call to simplified_parser_processor->process_chunk()
line 190 of /backup/util/xml/parser/progressive_parser.class.php: call to progressive_parser_processor->receive_chunk()
line 278 of /backup/util/xml/parser/progressive_parser.class.php: call to progressive_parser->publish()
line ? of unknownfile: call to progressive_parser->end_tag()
line 179 of /backup/util/xml/parser/progressive_parser.class.php: call to xml_parse()
line 158 of /backup/util/xml/parser/progressive_parser.class.php: call to progressive_parser->parse()
line 110 of /backup/util/plan/restore_structure_step.class.php: call to progressive_parser->process()
line 181 of /backup/util/plan/base_task.class.php: call to restore_structure_step->execute()
line 210 of /backup/moodle2/restore_activity_task.class.php: call to base_task->execute()
line 178 of /backup/util/plan/base_plan.class.php: call to restore_activity_task->execute()
line 167 of /backup/util/plan/restore_plan.class.php: call to base_plan->execute()
line 339 of /backup/controller/restore_controller.class.php: call to restore_plan->execute()
line 224 of /backup/util/ui/restore_ui.class.php: call to restore_controller->execute_plan()
line 135 of /backup/restore.php: call to restore_ui->execute()
 
Average of ratings: -
Picture of David Vogel
Re: Restore error: duplicate entry in mdl_forum_subscriptions
 

As a test, I deleted the mdl_forusubs_usefor_uix index and the restore worked properly. After the restore, I do see multiple duplicate entries in mdl_forum_subscriptions, which is confusing to me because I don't see how I can back up a course with no rows with values U-F in mdl_forum_subscriptions and then restore it and find two rows with values U-F (U and F representing the duplicate user/forum ID pairs). 

 
Average of ratings: -
Picture of David Vogel
Re: Restore error: duplicate entry in mdl_forum_subscriptions
 

In case you're wondering if I somehow missed duplicate entries in the original course, it's still in development so the mdl_forum_subscriptions table is very small. These are all the entries before the course restore (20 total). After restoring the course with the unique mdl_forusubs_usefor_uix index deleted, I have 47 entries, including several duplicate pairs like 6-130 which didn't exist in the original table even once. 

phpMyAdmin view


EDIT:

Huh. I went looking, and I don't even have forum IDs above 124. However, after the restore I've got forum values in mdl_forum_subscriptions as high as 158. All the duplicate entries have forum ID values above 124, which means they don't correspond to any actual forums that exist in the site. I wonder if the automatic restore process has a bug where it's putting the data for something else (not forum subscriptions) into mdl_forum_subscriptions? 

 
Average of ratings: -
Picture of David Vogel
Re: Restore error: duplicate entry in mdl_forum_subscriptions
 

Okay, one last post. I've tried the restore a few times and have watched the forum ID values. A few more observations: 

1. After each restore, all the new forum ID values are over 124, that is, higher than the ID of any forum which actually exists on the site. 

2. The new forum ID values in mdl_forum_subscriptions seem to be incrementing each successive time I try a restore with the same .mbz file. At first they were in the 120s and 130s, then 140s and 150s, and the most recent restore has new forum ID values in the 160s and 170s. 

3. The duplicate forum IDs seem to be the same. Last time, all the users with ID 6 or above had two forum subscriptions with forum ID 145. In the most recent restore, those same users had two forum subscriptions with forum ID 160. 


None of this makes sense to me, but I'm going to keep digging. I'm hoping someone else with more knowledge of the underlying code may have more insight into what might be happening? 

 
Average of ratings: -
Picture of Ken Task
Re: Restore error: duplicate entry in mdl_forum_subscriptions
Particularly helpful Moodlers

Can' t say  I've had the same exact issue, but can share what might help a little in discovering what might be going on.   When a course is restored, Moodle uses the moodledata/temp/backup/ directory.  It first un-archive's the .mbz into a hash named directory.   xml files with matched directories should be present ... example: files.xml there should be a files directory.   The most import xml file is moodle_backup.xml as it serves as a 'roadmap' for the 'plan' moodle builds on a temporary basis to restore the course.

I don't have a backup with lots of forums to inspect to see which xml points to what directory for forums ... suspect it's activities.

I have sent an offer via PM ...

'spirit of sharing', Ken

 
Average of ratings: -
Picture of Ken Task
Re: Restore error: duplicate entry in mdl_forum_subscriptions
Particularly helpful Moodlers

This line looks like it's suspect:

line ? of unknownfile: call to progressive_parser->end_tag()
That whole routine looks like it's working on an xml file ... a road map for the forums contained in the backup.   The progressive_parser thang.

The fact you are seeing dups in DB tables might not be surprising as there are many tables in moodle DB that are incremental ... just adds a row.   And since restore fails, moodle has no way to know the tables it created to that point are dups/errors.

Wonder ... in 3.4 there are task for cleaning up temp areas of Moodle data.  Cron runs the task and cron supposedly set to run every minute now ... but the task have their own schedules ... when to run.

Can turn off a task ... on a temporary basis ... try the restore again.   Of course, if it fails, then inspect what's remaining in moodledata/temp/backup/ ... probably the same as their other hashed named backup directories that remain ... providing not much of any clue, but at this point, ANY clue would be nice.

Uhhhh ... take me up on that offer made via PM!   In case anyone is wondering the 'offer' doesn't involve any fee ... just my time. ;)

'spirit of sharing', Ken


 
Average of ratings: -
Picture of David Vogel
Re: Restore error: duplicate entry in mdl_forum_subscriptions
 

Ken-- thanks very much for the additional information and offer to help. 

For anyone else who's reading this thread, one point of clarification and correction. In the posts above, I stupidly forgot that the restore itself would be creating new forums (duh!). Thus, I was wrong above that it's creating IDs for forums which don't exist. The new forums are created during the restore and then subscriptions are linked to them in mdl_forum_subscriptions. However, there's still the issue of the repeated duplicate subscriptions. (The same thing is happening in multiple courses and on my dev Moodle.) 

I'm thinking this patch will fix the issue: https://tracker.moodle.org/browse/MDL-60669  I installed 3.4 a few days before the patch was integrated, so I'll need to update. I'll post here if that fixes it. Still frustrating not to know what's causing this, though. 

 
Average of ratings: -
Picture of David Vogel
Re: Restore error: duplicate entry in mdl_forum_subscriptions
 

I'm still not sure what caused the duplicate subscriptions, but https://tracker.moodle.org/browse/MDL-60669 fixes the issue. Once I updated my 3.4 code to a version after the patch was integrated into the main code (Dec 1), I was able to restore successfully without any duplicate subscription errors. 

 
Average of ratings: -