After a lot of work, I was able to create a backup file from the command line from a HUGE course I need to move from one installation of moodle to a new one.
The backup file is very, very large (2.2GB), so doing it via the web-form is not feasible. Luckily, moosh came to the rescue. Or so I thought.
Everytime I try to restore this course, either on an existing course or creating a new course, I receive the same error:
Top Moodle dir: /var/www/sitename.com/moodle
Current working dir: /var/www/sitename.com/moodle
Relative Moodle dir:
Processing command option 'help''
Processing command option 'directory''
Processing command option 'existing''
Processing command option 'overwrite''
Command: restore (course)
Options:
help (): ''
directory (): ''
existing (): ''
overwrite (): ''
Arguments:
../backup-moodle2-course-77-[100_%_xxx_xxx_xxx]_1-_fundamental_(feb._28_-_may_9_)-20170628-1354-nu.mbz 6
Extracting Moode backup file to: '/var/www/moodledata/temp/backup/moosh_restore_5953d23ceb530'
Creating new course to restore backup
instantiating restore controller 8d7fd7eac847a8a62d5adae0672d2db9
setting controller status to 100
loading backup info
loading controller plan
setting controller status to 300
applying restore defaults
Unknown setting: questionbank
setting controller status to 400
checking plan security
setting controller status to 600
saving controller to db
calculating controller checksum 6e081c618644174d06d48a520580a14f
loading controller from db
Restoring (new course id,shortname,destination category): 135,[100 % XXX XXX XXX] 1- FUNDAMENTAL (Feb. 28 - May 9 )_2,6
Default exception handler: error/setting_by_name_not_found Debug:
Error code: setting_by_name_not_found
$a contents: enrol_migratetomanual
* line 124 of /backup/util/plan/base_plan.class.php: base_plan_exception thrown
* line 97 of /backup/util/plan/base_task.class.php: call to base_plan->get_setting()
* line 159 of /home/ubuntu/moosh/Moosh/Command/Moodle29/Course/CourseRestore.php: call to base_task->get_setting()
* line 285 of /home/ubuntu/moosh/moosh.php: call to Moosh\Command\Moodle29\Course\CourseRestore->execute()
++ This page should be using theme leapglobal which cannot be initialised. Nor can the site theme leapglobal. Falling back to boost ++
* line 516 of /lib/outputlib.php: call to debugging()
* line 1596 of /lib/pagelib.php: call to theme_config::load()
* line 1848 of /lib/setuplib.php: call to moodle_page->initialise_theme_and_output()
* line 388 of /lib/setuplib.php: call to bootstrap_renderer->__call()
* line ? of unknownfile: call to default_exception_handler()
!!! error/setting_by_name_not_found !!!
!!
Error code: setting_by_name_not_found
$a contents: enrol_migratetomanual !!
!! Stack trace: * line 124 of /backup/util/plan/base_plan.class.php: base_plan_exception thrown
* line 97 of /backup/util/plan/base_task.class.php: call to base_plan->get_setting()
* line 159 of /home/ubuntu/moosh/Moosh/Command/Moodle29/Course/CourseRestore.php: call to base_task->get_setting()
* line 285 of /home/ubuntu/moosh/moosh.php: call to Moosh\Command\Moodle29\Course\CourseRestore->execute()
!!
The error seems to be complaining about not finding "questionbank" settings, but I don't understand WHERE it is not finding that setting.
I have also found this thread, and tried applying the suggested fix, to no avail.
Hitting a wall with this, and we really need to be able to export/import this course (variations of this course are created seasonally, and they always start using a copy from the previous course as a template).
Thanks in advance for anything that pushes us in the right direction,
Kind regards
You've got one thing in your favor ... running linux! ;)
Qualifications for this response ... help admin a server that has had 'larger' courses ... from 20+ Gig up to and including (believe it or not) a 139Gig course ... (don't ask!!!!).
Using the command line backup.php script in moodlecode/admin/cli/ was successful in backing up those large courses ... one at a time. Note the backup script takes preferences from Moodle Admin UI setup of what to backup as it's defaults and no way to change those via command line (I'd remove stats and logs in preferences for backups for a big course backup run - then turn them back on later).
The 'trick' ... is to use 'nohup' in front of the php command to run the backup.php script.
Since I see Ubutnu in your output of moosh, here's a reference:
http://manpages.ubuntu.com/manpages/trusty/man1/nohup.1.html
from moodlecode/admin/cli/ looks like:
nohup php ./backup.php --courseid=INTEGER --destination=STRING
The destination was to a partition that had enough space for the approximate size moosh info would provide ... so --destination=/mnt/data/moodlebackups/ was my destination.
For that 139Gig course to backup had to shutdown apache service to gain enough memory so that php and mySQL were the only apps involved - and with nohup it ran almost all night long.
I use CentOS or RHEL and have used nohup on those flavors. Works.
I don't see anything in the error related to questionbank, but do see mention of a theme.
Is that theme installed/compat with the version of Moodle to which the large backup is being restored via Moosh?
Since you mentioned your process ... the course backed up then restored to begin a new fresh course with same content ... are your resetting the course first ... selecting options to remove all student and student work from the course leaving only the teacher?
'spirit of sharing', Ken
Hi,
Just about the fact that the backup script uses UI defined options : some times ago, i created MDL-52050 to ask for an evolution of this script, to allow it to use specific parameters. Don't hesitate to vote / comment / developp for it
Séverin
One more item ...
During backups or restores Moodle uses moodledata/temp/backup/
as a build area. In some cases, Moodle actually uncompresses the .mbz backup to rebuild a compatible backup to restore.
The build directory is a hash name ... like a check sum number seen in your output. On a failed backup or restore the build directory remains ... not cleaned up until the clean up task in the cron job runs. So for a little while one can inspect stuff.
With moosh: moodledata/temp/backup/moosh_restore_5953d23ceb530
Inside that directory one should find .xml files which are roadmaps to things ... like quizzes/test. There is a quiz or questions.xml flie which can be opened with a text editor and inspected. One could use a find command in nano or any text editor for any settings related to questionbank to get a clue/hint.
Just a thought. ;)
'spirit of sharing'. Ken
Ok, I lied! One/Two more thoughts ... things to try ...
http(s)://site/admin/tool/health/
no parameters to it ... script just runs. Could be heavy processing so execute during non-prime time if you can.
What it might discover (and has when I've used it) ... issues with question bank. If it can, health will make suggestions (sql commands) one could run via a mysql session with DB.
On several sites I have seen it's recommendations fix things related to questionbank.
Also, there is a cli script called fix_orphaned_question_categories.php one could run.
Run both of those before making the backup to restore, of course.
Ok, now I am out of ideas. ;)
'spirit of sharing', Ken
Thanks a lot Ken.
Just got to the office. I need to get on top of a few emails, and later I'll get through the troves of information in your messages.
Thansk again, I'll let you know in a couple hours how did it go!
Regarding the "questionbank" thing:
In the error I see:
applying restore defaults
Unknown setting: questionbank
setting controller status to 400
Later on it says, reading the trace:
Error code: setting_by_name_not_found
$a contents: enrol_migratetomanual !!
!! Stack trace: * line 124 of /backup/util/plan/base_plan.class.php: base_plan_exception thrown
* line 97 of /backup/util/plan/base_task.class.php: call to base_plan->get_setting()
* line 159 of /home/ubuntu/moosh/Moosh/Command/Moodle29/Course/CourseRestore.php: call to base_task->get_setting()
* line 285 of /home/ubuntu/moosh/moosh.php: call to Moosh\Command\Moodle29\Course\CourseRestore->execute()
I see that the setting_by_name_not_found exception is thrown on get_setting(), and that is thrown because "questionbank" setting is not found, hence my thought that that could be involved.
Uninstalled the 'leapglobal' theme, and now I no longer receive that notice about the missing theme, but the error persist:
Creating new course to restore backup
instantiating restore controller 0ed695ef1e34684a49ab61f5aa845a35
setting controller status to 100
loading backup info
loading controller plan
setting controller status to 300
applying restore defaults
Unknown setting: questionbank
setting controller status to 400
checking plan security
setting controller status to 600
saving controller to db
calculating controller checksum 73d1f3d8b9dae13257bb1b8bfa0b17e4
loading controller from db
Restoring (new course id,shortname,destination category): 132,[100 % PROGRAMMATIC] 1- FUNDAMENTALS (Feb. 28 - May 9 )_1,6
Default exception handler: error/setting_by_name_not_found Debug:
Error code: setting_by_name_not_found
$a contents: enrol_migratetomanual
* line 124 of /backup/util/plan/base_plan.class.php: base_plan_exception thrown
* line 97 of /backup/util/plan/base_task.class.php: call to base_plan->get_setting()
* line 159 of /home/ubuntu/moosh/Moosh/Command/Moodle29/Course/CourseRestore.php: call to base_task->get_setting()
* line 285 of /home/ubuntu/moosh/moosh.php: call to Moosh\Command\Moodle29\Course\CourseRestore->execute()
!!! error/setting_by_name_not_found !!!
!!
Error code: setting_by_name_not_found
$a contents: enrol_migratetomanual !!
!! Stack trace: * line 124 of /backup/util/plan/base_plan.class.php: base_plan_exception thrown
* line 97 of /backup/util/plan/base_task.class.php: call to base_plan->get_setting()
* line 159 of /home/ubuntu/moosh/Moosh/Command/Moodle29/Course/CourseRestore.php: call to base_task->get_setting()
* line 285 of /home/ubuntu/moosh/moosh.php: call to Moosh\Command\Moodle29\Course\CourseRestore->execute()
!!
Ran health-check, no problems found. Executed fix_orphaned_question_categories.php: "No orphaned question categories found".
Re-exported the course, Re-attempted to restore. Same result.
Creating new course to restore backup
instantiating restore controller 0ed695ef1e34684a49ab61f5aa845a35
setting controller status to 100
loading backup info
loading controller plan
setting controller status to 300
applying restore defaults
Unknown setting: questionbank
setting controller status to 400
checking plan security
setting controller status to 600
saving controller to db
calculating controller checksum 73d1f3d8b9dae13257bb1b8bfa0b17e4
loading controller from db
Restoring (new course id,shortname,destination category): 132,[100 % PROGRAMMATIC] 1- FUNDAMENTALS (Feb. 28 - May 9 )_1,6
Default exception handler: error/setting_by_name_not_found Debug:
Error code: setting_by_name_not_found
$a contents: enrol_migratetomanual
* line 124 of /backup/util/plan/base_plan.class.php: base_plan_exception thrown
* line 97 of /backup/util/plan/base_task.class.php: call to base_plan->get_setting()
* line 159 of /home/ubuntu/moosh/Moosh/Command/Moodle29/Course/CourseRestore.php: call to base_task->get_setting()
* line 285 of /home/ubuntu/moosh/moosh.php: call to Moosh\Command\Moodle29\Course\CourseRestore->execute()
!!! error/setting_by_name_not_found !!!
!!
Error code: setting_by_name_not_found
$a contents: enrol_migratetomanual !!
!! Stack trace: * line 124 of /backup/util/plan/base_plan.class.php: base_plan_exception thrown
* line 97 of /backup/util/plan/base_task.class.php: call to base_plan->get_setting()
* line 159 of /home/ubuntu/moosh/Moosh/Command/Moodle29/Course/CourseRestore.php: call to base_task->get_setting()
* line 285 of /home/ubuntu/moosh/moosh.php: call to Moosh\Command\Moodle29\Course\CourseRestore->execute()
!!
Ran health-check, no problems found. Executed fix_orphaned_question_categories.php: "No orphaned question categories found".
Re-exported the course, Re-attempted to restore. Same result.
And regarding the course-reset... I can't really reset the course data, since they use for stats and whatnot.
But, in my default backup settings I have set it up so it only exports:
backup_general_activities
backup_general_questionbank
All other settings are disabled...
Updated to latest 3.3+, same result. (my version was from two-three weeks ago).
It's at this stage (*****) below:
setting controller status to 600
saving controller to db
calculating controller checksum 73d1f3d8b9dae13257bb1b8bfa0b17e4
loading controller from db
*****
Restoring (new course id,shortname,destination category): 132,[100 % PROGRAMMATIC] 1- FUNDAMENTALS (Feb. 28 - May 9 )_1,6
Default exception handler: error/setting_by_name_not_found Debug:
*****
where it fails ... from what I see.
Moosh appears to be attempting to restore using shortname and destination category.
What moosh command are you using to restore? Exactly.
I see on
http://moosh-online.com/commands/
only two.
Have you tried both?
Know the following would be a pain, but ... what if one created a blank course first ... in the category desired ... and then restored the backup to that course.
'spirit of sharing', Ken
One other idea ...
The file name:
backup-moodle2-course-77-[100_%_xxx_xxx_xxx]_1-_fundamental_(feb._28_-_may_9_)-20170628-1354-nu.mbz
That's a doozy of a course title and/or shortname.
Moosh nor Moodle Web based restore routine doesn't care about file names, but contained in the backup there is data for course fullname and shortname .... just like above.
This wouldn't be hard to try:
un gzip the backup file.
In the moodle_backup.xml file, shorten the 'shortname' taking out characters like % and ().
Save the xml file.
Repackage the backup (gzip with .mbz extension) ... this time give it backuptest1.mbz as a name.
Try to restore.
Since the backups contain no users, am about to suggest you upload the .mbz file where I can acquire it so I can un-archive and inspect the contents ... as well as attempt restores on a similar machine.
'spirit of sharing', Ken
I'm using `moosh course-restore`, not aware of any other command.
I'm afraid I can't send out the mbz file. The course data is proprietary and I'm strictly forbidden to send outside the company. That is something that is taken very seriously around here.
I don't think the shortname is the real problem, since the new course actually gets created, albeit completely empty. And I had figured that the filename wasn't really relevant, but that was what created by the native backup.php. I've used a more rational name when using `moosh course-backup`, same result.
Also, I've tried changing the shortname to something more sensible and attempted to restore: same result.
And I've tried to restore into an existing course (either empty or not empty): same result.
Or I'm going to put the server to fire. My first experience with Moodle hasn't been entirely happy so far.
Thanks for your help anyway.
Yes, check out the moosh site for the two ...
http://moosh-online.com/commands/
You have not shared what command you are using, even though asked. Come on now ... am trying to help blindly here (can only guess from information provided).
Ok, I can't get a backup to inspect ... sigh ... bet also your network/IT people frown on remote access as well?
Ahhh ... something new ... the course does get restored ... just no content. Has it always done that? or is this the first time for that?
Hmmmm ... might have to increase PHP settings for time for script to run, memory a script can consume, etc.
Have you checked other server logs ... error log for apache, error log for php ... there might be php_error files in directories of Moodle code ... which could give you very targeted rabbits to chase.
Debugging on is good ...
Yes, first experiences admin'ing a moodle are usually not pleasant ... Moodle not like WordPress or Joomla or other such. 'Not in Kansas anymore, Toto'! kinda thing. Like a fungus, however, it might grow on ya. ;)
'spirit of sharing', Ken
Ken, thanks for your help, but I did say which command I'm using: moosh course-restore. And I don't know of any other command useful to restore a backup.
It always behave the same way: creates an empty course.
Not about memory or timeout, and not dependant on a particular server (happens both on the production machine, dev machine, local dev machine, etc) it is clearly a php exception.
The exception is thrown because it can't find the 'enrol_migratetomanual' setting.
On CourseRestore.php:158-161 (on the moosh side) we have this:
foreach ($tasks as &$task) {
$setting = $task->get_setting('enrol_migratetomanual');
$setting->set_value('1');
}On the first iteration, when getting the 'enrol_migratetomanual' for the 'root_task' (whatever that is), it ends up calling get_setting() on the base_task class.It fails to find the setting both on the "task" and on the "plan", and throws and exception, which bubbles up and shorts exeution.
I guess that the question is: what is this setting, and how and where can I set it so this restore task works properly???
Problem seems to be moosh related (disregarding the fact that moodle doesn't provide a command line solution to restore backups!!!!).
Was asking which of these:
course-restore
Restore course from path/to/backup.mbz to category or existig course.
Example 1: Restore backup.mbz into category with id=1
moosh course-restore backup.mbz 1
Example 2: Restore backup.mbz into existing course with id=3
moosh course-restore -e backup.mbz 3
Guess you've found this?
https://github.com/tmuras/moosh/issues/189
about the issue. BTW, the command used is shown in the above.
'spirit of sharing', Ken
Those aren't different commands, but different options for the same comand.
And yes, I said I tried both.
I said:
>> And I've tried to restore into an existing course (either empty or not empty): same result.
(Which means using the "-e" parameter).
Also tried to use the "-o" parameter, for which I couldn't find documentation but that apparently controls if the command overwrites the content of the course, or adds content to the course. Same result.
Yes, I found that moosh issue. It was linked in my previous comment, where I say it seems to be moosh related.
I patched a flimsy workaround in moosh and I was able to restore the backup. Details are posted in the linked github issue.
Thanks for your help anyway.