I backed up everything from the 6 courses through the web interface in Moodle 2.2 this morning. I'm definitely working from the back version which has user data in it (I checked the mbz file and looked in users.xml).
When I restore the course is restored but the activities, groups and users are not. I did a previous restore which I've tried to overwrite. I've used the Moodle 3.2 web interface and Moosh (using latest build of the course-restore). I've run the cron to tidy up. The users have been created at some point but they aren't being enrolled in the courses.
What I want to end up with is these 6 courses with activities, groups and users as they are in the old system. The other courses don't need users or groups.
Please can someone help me to think what may be the problem?
Thanks Ken. That's interesting.
Is it possible that the groups & groupings are trying to restore to specific unique ids in each respective database table? I'm seeing specific ids in groups.xml (part of the mbz file):
<?xml version="1.0" encoding="UTF-8"?>
If I could force it to insert those groups then it might work. I presume that Moodle won't overwrite groups for obvious reasons.
Uhhh ... like I said ... 'guess'. ;) Servers I support don't use groups/groupings nor cohorts ... neither fits their needs. Fact that you are seeing something related in backup file xml's suggest IF those groups/groupings existed in the new server, then users would be imported with that info about them ... and hopefully enrolled in courses desired.
Above is 'human logic' .. or at least mine ... which isn't the same as 'Moodle logic' ... for sure! ;)
'spirit of sharing', Ken
This still isn't working, but I've found some more information. I'd be grateful for any suggestions
The problem: course restore does not complete. It times out in the browser but continues working at the server. The server session terminates but I can't find out why.
- increasing the timeouts in PHP, Apache and MySQL.
- increasing the MySQL settings (wait_timeout, query_cache_limit,query_cache_size and
- truncating the tables mdl_backup_controllers and mdl_backup_logs
- removing old backups in moodledata/temp/backup
- restoring parts of the course at a time and mergeing
I've observed that:
- the database session continues after the browser times out. This can be seen using SHOW FULL PROCESSLIST;. The session is loading users into a temp table, but the table never grows in size, suggesting that the data is cached. This may be the problem
- The backup file, when uncompressed, contains a quiz whose contents are 338M in size. The quiz in question has 7600 enrolled users and 5600 quiz attempts. It seems to be this big file which fails
- this course will restore to a fresh database on my vagrant Virtual Machine.
Does anyone have any hints about how to solve this?
I hope you are making the db tweaks with some information from something like MySQLTuner ... sometimes one can get a point where config changes for DB upwards actually begets diminishing returns.
I'll give it another 'guess' ....
Is the 2.2 site still up and can you get to that course there?
If so, reset that course ... that will remove students *and their work* ... quizzes and quiz attempts.
Make no user backup of that reset course - just to make sure nohting of students is contained in the backup.
Since you've learned how to inspect a backup, un-compress and look at the xml flies ... sizes.
Must say that's one large 'quiz'!!!! :\
Something else if the 2.2 site is still up .... can you export the quiz bank?
Am also gonna stick my neck out and say you're only hope might be moosh from command line. Takes browser out of loop ... apache out of loop ... just PHP and DB server then.
Will send a PM with some other strategy to assist.
'spirit of sharing', Ken
Pardon the follow up ... sometimes one has to read things more than once ... 'this course will restore to a fresh database on my vagrant Virtual Machine.'
What are specs of this 'vagrant VM'? like os platform, memory, processors, etc.
compared to the server that's having issues.
In that Vagrant VM' ... does that have a moodle 3.2 installed on it or are you just importing something to the DB server? importing what?
'spirit of sharing', Ken
The vagrant specs are default php7 and mysql 5.7. You can see the build script here:
As it happens, there are two monster courses. The first did restore to the VM. The second just failed. It expands to 375Mb, with >5000 enrolled students and roughly the same number of attempts. This other course generated a dmlwriteexception error.
Course reset > I've succeeded using the backup file but excluding the quiz. It's not the backup file which is at fault. I don't think that the reset option is going to solve this.
export the quiz bank > I haven't tried this. I'll look at it. I need the attempts to come through too though. I'm assuming that only the quiz structure comes through with an export.
Moosh > I have made good use of Moosh for restoring other courses on this server. It's been very useful. In the case of these 2 courses it didn't succeed. I've tried it again. As I write I can see this MySQL process:
Id User Host db Command Time State Info
196 moodle localhost moodle Query 0 query end INSERT INTO mdl_backup_ids_temp (backupid,itemname,itemid,newitemid,parentitemid,info) VALUES('....
I can watch the same MySQL process ID running different queries. In this instance it is supposed to be loading into table mdl_backup_ids_temp but the table never increases in size. Eventually the restore exits with the error ans warnings
Restore pre-check failed!
string(174) "The questions category "Default for level 2", originally at system/course category context in backup file, will be created at course context by restore"
which doesn't tell me anything useful.
I looked in vain for a server side problem. I then did the restore using Google Chromium browser (Chrome) and it worked. Mozilla Firefox continues to time out. I'm not sure why Firefox doesn't keep the session alive in the browser.
The session continues operating on the database after the browser times out. That's why I kept looking for a server side problem.
Here are some other useful diagnostics which I found on the way:
- Show the queries being processed by MySQL by passing
SHOW FULL PROCESSLIST;which is a direct command whose equivalent is:
SELECT * FROM INFORMATION_SCHEMA.PROCESSLIST;
- You can see the size of tables (and their changes) by passing
SELECT TABLE_NAME, TABLE_ROWS FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_NAME LIKE 'mdl_%';
- Turn on the MySQL general log by adding this to your my.cnf file in the [mysqld] section (from StackOverflow)
general_log = on
- restores where there are a very large number of users and attempts take a significant period of time. The course which caused this forum thread generated 1,989,000 queries. The quiz.xml file which contained the data was 323MB when unpacked.
Glad to hear you've resolved the issue ... about your PM concerning submitting to Tracker. May as well. And thanks for providing exactly what you did ... there's a shared query for custom_sql addon that shows largest tables, BTW, and have had to use it in troubleshooting issues on sites before.
Guess there is a reason that quiz attempts logs were kept. The quiz.xml file is asciii and 323 MB does sound like a lot of questions.
If the old site still up, there is a hidden admin tool to check the health of Moodle.
If there are issues with quizzes the health check does report them as well as possible fixes via SQL commands/statements one could run via CLI or any tool that could execute SQL on the DB for the Moodle.
Wonder what that might report.
'spirit of sharing', Ken