I still can't get my course to backup on my noodle 2.6 installation and then restore successfully to a brand new 2.9 installation.
(Edited by Mary Cooch - original submission Tuesday, 16 June 2015, 1:52 AM)
I am not willing to take any money. How big is your backup? If it is not too big, I can try restoring it in one of my 2.9 versions of moodle, of course, keeping everything confidential. I cannot guarantee success.
Is it less than 10MB? 20MB?
If I succeed, I might then be able to make a backup in 2.9 and give it back to you.
To get this big of a file to me, I would probably have to download it from your web server. Too big for email. Or you could get it to me in some other way.
(Edited by Mary Cooch to change title- original submission Tuesday, 16 June 2015, 2:42 AM)
Rick,
Thanks for your help. The course backup is 380 MB. This, of course, may well be the problem. I have avoided uploading video to the course, but I do have over 2500 questions, many that have multiple images. To reduce the size of the backup, I have reset the course to remove any quiz attempts, submissions, enrolled users, etc. I also set the backup to not backup any user information.
I have placed the file on dropbox at the following URL. (It might take a while for it to actually upload to dropbox)
I am going to give it another try this morning after upping all the limits on my server.
execution limit = 99999
memory limit = 99999
max input variables = 99999
Max input time =9999
I will tell you how it goes....
5 minutes later, the following is displaying...
I am not sure why it would say 28 thousand seconds have gone by...
I have a small VPS for my small Moodle. For me, course restores (10MB in size) can take 10 to 15 minutes. I just let it happen. So for your 380MB file, restoring it could take awhile. I am not sure what kind of server you have.
Rick,
I also have a small VPS server. I am hosted a Cloudways. They have been excellent and have 1 click moodle 2.9 installation.
There is nothing else going on on this server aside from this one class. I have another server that has 2GB ram and it seems to hold up well with 3 classes running even if 25 students are actively clicking away on a quiz. (In my experience, if the quiz module won't crash the site, pretty much nothing will. )
I also have a small VPS server. I am hosted a Cloudways. They have been excellent and have 1 click moodle 2.9 installation.
There is nothing else going on on this server aside from this one class. I have another server that has 2GB ram and it seems to hold up well with 3 classes running even if 25 students are actively clicking away on a quiz. (In my experience, if the quiz module won't crash the site, pretty much nothing will. )
I didn't take a screen shot when I restored your course, but my server did the same thing with your course ... uhhhhh, I waited a lot longer than you did. Matter of fact, did some other chores, left it running at that point and returned about 30 minutes later to see the 'progress bar' did start progressing.
So it could be that you just didn't wait long enough?
See other posting concerning your course from HELL below ...
'spirit of sharing', Ken
Yes, a very big backup.
In my php.ini file, I have memory_limit - 256M. I wonder if the "M" in this is significant. After you make these changes, have you verified that they are really in effect (by running info.php)?
If you have the ability to redo the backup, consider choosing the smallest set of options for the backup (e.g., no students, no question bank, etc.) In other words, start with the smallest backup and work your way upwards.)
I also wonder if you have to change: upload_max_filesize, and post_max_size.
If I eliminate the question bank from the restore, it will restore properly.
Unfortunately, when I transfer the question bank over using export and then import, all of my quizzes are broken.
This is how I managed to get it to work when I upgraded from 2.4 to 2.6, but I had hoped that that solved the problem permanently. It seems that the problem is back. I hope to solve it once and for all so that I do not have to revisit this issue again in the future.
I am sort of hoping that it is the size of the backup that is causing the issue as I imagine that this could be addressed through server settings.
(I did try to reduce the backup file size by including the question bank but only having it back up the very first unit of study. This got the size down to 50 MB. It still would not restore properly. )
Let me try running info.php and see what it says.
I have an ftp connection up and running. Which file to I go into to change the upload_max_filesize, and post_max_size? If memory servers me correctly, there are multiple places that might need to be changed???
So, just unchecking the question bank allows this to restore. Interesting. Maybe Tim Hunt will provide some ideas. He might also have some ideas about what is happening to your images.
upload_max_filesize and other settings are in your php.ini file. I believe there is only one php.ini that controls your moodle, but yes, you might find several of these scattered around on your server. You have to find the right one. I forget the easiest way (via linux command) to do this. FTP would be the wrong approach, and instead, this is best done with a terminal program like Putty (and many others), or a text editor (many) on your server.
By the way, here are the apache error log messages for this failed attempt at installing..
[Tue Jun 16 14:28:51 2015] [error] [client 65.186.77.14] instantiating restore controller 97223f272b24319bfd108819819461ae, referer: http://moodle-1840-18176-42202.cloudwaysapps.com/backup/restore.php
[Tue Jun 16 14:28:51 2015] [error] [client 65.186.77.14] setting controller status to 100, referer: http://moodle-1840-18176-42202.cloudwaysapps.com/backup/restore.php
[Tue Jun 16 14:28:51 2015] [error] [client 65.186.77.14] loading backup info, referer: http://moodle-1840-18176-42202.cloudwaysapps.com/backup/restore.php
[Tue Jun 16 14:28:52 2015] [error] [client 65.186.77.14] loading controller plan, referer: http://moodle-1840-18176-42202.cloudwaysapps.com/backup/restore.php
[Tue Jun 16 14:28:52 2015] [error] [client 65.186.77.14] setting controller status to 300, referer: http://moodle-1840-18176-42202.cloudwaysapps.com/backup/restore.php
[Tue Jun 16 14:28:52 2015] [error] [client 65.186.77.14] checking plan security, referer: http://moodle-1840-18176-42202.cloudwaysapps.com/backup/restore.php
[Tue Jun 16 14:28:52 2015] [error] [client 65.186.77.14] setting controller status to 500, referer: http://moodle-1840-18176-42202.cloudwaysapps.com/backup/restore.php
[Tue Jun 16 14:28:52 2015] [error] [client 65.186.77.14] checking plan security, referer: http://moodle-1840-18176-42202.cloudwaysapps.com/backup/restore.php
[Tue Jun 16 14:28:52 2015] [error] [client 65.186.77.14] checking plan security, referer: http://moodle-1840-18176-42202.cloudwaysapps.com/backup/restore.php
[Tue Jun 16 14:28:52 2015] [error] [client 65.186.77.14] checking plan security, referer: http://moodle-1840-18176-42202.cloudwaysapps.com/backup/restore.php
[Tue Jun 16 14:28:52 2015] [error] [client 65.186.77.14] saving controller to db, referer: http://moodle-1840-18176-42202.cloudwaysapps.com/backup/restore.php
[Tue Jun 16 14:28:52 2015] [error] [client 65.186.77.14] calculating controller checksum 2bb6932a643d9f1f7b7a5184962c4fb8, referer: http://moodle-1840-18176-42202.cloudwaysapps.com/backup/restore.php
[Tue Jun 16 14:29:13 2015] [error] [client 65.186.77.14] loading controller from db, referer: http://moodle-1840-18176-42202.cloudwaysapps.com/backup/restore.php
[Tue Jun 16 14:29:13 2015] [error] [client 65.186.77.14] checking plan security, referer: http://moodle-1840-18176-42202.cloudwaysapps.com/backup/restore.php
[Tue Jun 16 14:29:13 2015] [error] [client 65.186.77.14] checking plan security, referer: http://moodle-1840-18176-42202.cloudwaysapps.com/backup/restore.php
[Tue Jun 16 14:29:13 2015] [error] [client 65.186.77.14] checking plan security, referer: http://moodle-1840-18176-42202.cloudwaysapps.com/backup/restore.php
[Tue Jun 16 14:29:13 2015] [error] [client 65.186.77.14] saving controller to db, referer: http://moodle-1840-18176-42202.cloudwaysapps.com/backup/restore.php
[Tue Jun 16 14:29:14 2015] [error] [client 65.186.77.14] calculating controller checksum 2bb6932a643d9f1f7b7a5184962c4fb8, referer: http://moodle-1840-18176-42202.cloudwaysapps.com/backup/restore.php
[Tue Jun 16 14:29:50 2015] [error] [client 65.186.77.14] loading controller from db, referer: http://moodle-1840-18176-42202.cloudwaysapps.com/backup/restore.php
[Tue Jun 16 14:29:50 2015] [error] [client 65.186.77.14] checking plan security, referer: http://moodle-1840-18176-42202.cloudwaysapps.com/backup/restore.php
[Tue Jun 16 14:29:55 2015] [error] [client 65.186.77.14] checking plan security, referer: http://moodle-1840-18176-42202.cloudwaysapps.com/backup/restore.php
[Tue Jun 16 14:29:55 2015] [error] [client 65.186.77.14] checking plan security, referer: http://moodle-1840-18176-42202.cloudwaysapps.com/backup/restore.php
[Tue Jun 16 14:29:55 2015] [error] [client 65.186.77.14] saving controller to db, referer: http://moodle-1840-18176-42202.cloudwaysapps.com/backup/restore.php
[Tue Jun 16 14:29:55 2015] [error] [client 65.186.77.14] calculating controller checksum 2c53b262845ce820f44bb555355c0b7c, referer: http://moodle-1840-18176-42202.cloudwaysapps.com/backup/restore.php
[Tue Jun 16 14:30:58 2015] [error] [client 65.186.77.14] loading controller from db, referer: http://moodle-1840-18176-42202.cloudwaysapps.com/backup/restore.php
[Tue Jun 16 14:30:58 2015] [error] [client 65.186.77.14] checking plan security, referer: http://moodle-1840-18176-42202.cloudwaysapps.com/backup/restore.php
[Tue Jun 16 14:31:05 2015] [error] [client 65.186.77.14] checking plan security, referer: http://moodle-1840-18176-42202.cloudwaysapps.com/backup/restore.php
[Tue Jun 16 14:31:05 2015] [error] [client 65.186.77.14] setting controller status to 600, referer: http://moodle-1840-18176-42202.cloudwaysapps.com/backup/restore.php
[Tue Jun 16 14:31:05 2015] [error] [client 65.186.77.14] saving controller to db, referer: http://moodle-1840-18176-42202.cloudwaysapps.com/backup/restore.php
[Tue Jun 16 14:31:05 2015] [error] [client 65.186.77.14] calculating controller checksum 3e8deb8ebe582c4963cdfa6c7d63a89e, referer: http://moodle-1840-18176-42202.cloudwaysapps.com/backup/restore.php
[Tue Jun 16 14:31:05 2015] [error] [client 65.186.77.14] loading controller from db, referer: http://moodle-1840-18176-42202.cloudwaysapps.com/backup/restore.php
[Tue Jun 16 14:31:34 2015] [error] [client 65.186.77.14] setting controller status to 700, referer: http://moodle-1840-18176-42202.cloudwaysapps.com/backup/restore.php
[Tue Jun 16 14:31:34 2015] [error] [client 65.186.77.14] saving controller to db, referer: http://moodle-1840-18176-42202.cloudwaysapps.com/backup/restore.php
[Tue Jun 16 14:31:35 2015] [error] [client 65.186.77.14] calculating controller checksum 3f48f481de96aa3142862abb92d438ae, referer: http://moodle-1840-18176-42202.cloudwaysapps.com/backup/restore.php
[Tue Jun 16 14:31:35 2015] [error] [client 65.186.77.14] loading controller from db, referer: http://moodle-1840-18176-42202.cloudwaysapps.com/backup/restore.php
[Tue Jun 16 14:31:35 2015] [error] [client 65.186.77.14] setting controller status to 800, referer: http://moodle-1840-18176-42202.cloudwaysapps.com/backup/restore.php
For those following this 'mystery' ...
Guess on first error line above from observing the backend of a restore of this course:
97223f272b24319bfd108819819461ae looks like a hashed named directory which Moodle creates in
/moodledata/temp/backup/. and uses as a build area for restoring. Contained there in are the extracted directories and related .xml files ... as well as moodle_backup.xml which is the 'road map' (so to speak) for the restore process.
When I restored this course to another server it did take a long time ... and there was only one warning in logs:
PHP Warning: Unknown: Input variables exceeded 1000. To increase the limit change max_input_vars in php.ini. in Unknown on line 0, referer: http://site/backup/restore.php
"How many input variables
may be accepted. Use of this directive mitigates the possibility of
denial of service attacks which use hash collisions. If there are more
input variables than specified by this directive, an E_WARNING
is issued, and further input variables are truncated from the request.
This limit applies only to each nesting level of a multi-dimensional
input array."
Server to which it was restored:
CentOS Linux 5.11
Intel(R) Xeon(R) CPU E5520 @ 2.27GHz, 4 cores
Memory 16GB
PHP 5.5.25 (cli) (built: May 15 2015 07:47:31)
Copyright (c) 1997-2015 The PHP Group
Zend Engine v2.5.0, Copyright (c) 1998-2015 Zend Technologies
with Zend OPcache v7.0.4-dev, Copyright (c) 1999-2015, by Zend Technologies
with Xdebug v2.3.2, Copyright (c) 2002-2015, by Derick Rethans
So from what I read above and know that some providers of remotely hosted stuff actually hide such from customers that might be an issue for Curt. That's a guess.
Have the course restored on another server and invited Curt to login and check it out.
We'll see if I guessed right on what was 'doctored' in xml files before repackaging the backup and restoring.
'spirit of sharing', Ken
Do you have instances of "mod_assignment" in your old course? It may be the conversion to "mod_assign" that is taking a long time if you have many assignments with submissions etc.
Please confirm if this might be the problem, if so there are options for working around this.
Please confirm if this might be the problem, if so there are options for working around this.
Damyon,
Thanks for the reply. I took some steps to shrink the size of the course backkup. First, I "reset" the course to scrub it of any assignment submissions and quiz attempts. Also, during the backup, I set it to not back up any user data.
The size of the backup is 380 MB. (This is large, even though I have avoided uploading video to the course if at all possible. I have over 2500 questions, many of them using pictures.)
@curt ...
have downloaded your backup and extracted it on local laptop to inspect contents and extract files.
Findings - which may/may not 'fix', but ...
Rename the backup file name from:
backup-moodle2-course-5-r-phys_sci_(classroom)-20150614-2334-nu.mbz
to
backup-moodle2-course-5-r-phys_sci_classroom-20150614-2334-nu.mbz
i.e., exclude the ()'s. Future: never use ( or ) in a filename even if your Mac will let you! Same with "/" or any other reserved character in Debian (or whatever flavor of distro used to host your Moodle). Same goes for any "\" character ... 'Wonders' platform!
In the backup .mbz there is a file:
Observation_vs._Inference_(2013).ppt
The "(" and ")" in file names might be an issue.
Matter of fact, there were several files using "(" and ")" in the file names.
Making Quality Graphs (1) 2.ppt
move to Making Quality Graphs 1 2.ppt
Chap1 ALPHA Reading (1).pdf
Chap1 ALPHA Reading 1.pdf
Sample Homework for Format(1).pdf
Sample Homework for Format1.pdf
Paper Helicopter Instructions (1).doc
Paper Helicopter Instructions 1.doc
Files without filename extensions:
orbitals with no extension is a jpeg ... renamed to orbitals.jpg
lewis_dot_examples is a jpeg
lewis_dot_exampls is a jpeg
mod_assign reference in files.xml has
Miscellaneous/New Server - Bixel Online Physical Science/Ch 14 - CHARLIE Colors of Light Assignmen/Description/Chapter_14_Solutions.pdf
Was the file named with '/' characters contained in the file name?
or did the course actually have a Miscellaneous / New Server - Bixel Online Physical Sciene/Ch 14 - CHARLIE Colors of Light Assignmen / Description / directory structure?
Here's a clip of files.xml one reference:
<file id="56464">
<contenthash>48e4e3c0cfe9ac435e052c51556fe1d07f8795b4</contenthash>
<contextid>1827</contextid>
<component>question</component>
<filearea>answer</filearea>
<itemid>141057</itemid>
<filepath>/</filepath>
<filename>orbitals</filename>
<userid>2</userid>
<filesize>9423</filesize>
<mimetype>image/jpeg</mimetype>
<status>0</status>
<timecreated>1428851898</timecreated>
<timemodified>1428858336</timemodified>
<source>orbitals</source>
<author>curt bixel</author>
<license>allrightsreserved</license>
<sortorder>0</sortorder>
<repositorytype>$@NULL@$</repositorytype>
<repositoryid>$@NULL@$</repositoryid>
<reference>$@NULL@$</reference>
</file>
I see it's related to a question .. filename has no extension and but shows mime type.
image/jpeg
Command line fix to change:
<filename>orbitals</filename>
to
<filename>orbitals.jpg</filename>
OR, go back through the courses in the 2.6 site and inspect the images used in Questions.
And error there will be archived in the backup file ... when it comes to restoring ... it might cause confusion and thus time out.
Am editing all the little things found in files.xml right now and will rebuild an .mbz file and test restoring to a 2.9. Will get back with results.
BTW, one of the reasons am doing this is really selfish ... I do help ISD's with their Moodles and occassionally run into a course from HELL - when it comes to backups and restores. Have found that the only way to recover is to download and un-archive ... inspect the .xml files for the suspected issues ... fix/edit ... then manually re-archive to build the corrected backup.mbz and TIA! (Try It Again).
Not a fun method ... and might have to go through the process more than once ... but, eventually, get it right and am able to get back most of the course.
Will get back ...
'spirit of sharing', Ken
Ken, I am glad that you explained why you do this. I thought you do these things just for fun.
(Good old file naming problems...)
In a way it is 'fun' ... like solving a 'mystery' ... but hate when it's machine over man ... when it should be the other way round ... MAN OVER MACHINE!
Since I've probably been unsuccessful more times than successful (I learn the hard way!), I've a higher tolerance for failure - but at my age, it's wearing me down! :|
BTW, have been successful restoring what was a 2.7.1 course backup to a 2.9.new after 'doctoring' some .xml files and repackaging.
Took a long time! But don't have the time to find quizzes that have missing images, etc.. so have invited Curt (offline) to login to the site where restored so there are at least 2 sets of eyes on the issues.
'spirit of sharing', Ken
Ken,
I think we are onto something. I went and checked out my course that you uploaded to your website. Everything works perfectly. The images are showing in the questions and everything else works.
One interesting thing I noticed that might actually be related. If you notice at the top of my page there are some pictures and to the right of them are the names of all my units. These are actually links to the individual units. Clicking one hides all of the other units. It is similar to the jump menu, but the students find it easier to use. These seem to work just fine on your server. In the past, whenever I have restored a course, these links would point to the old server and I would have to go through them and fix them one at a time. I am not sure how these links managed to make it through the restore process, but they seem to.
I think that pretty much points to my server as the problem. Do you think I need to change one or more of the settings?
Cloudways has a very nice user interface I used to set the following settings:
In the second panel under MySQL - encoding is US ASCII - for a long time now Moodle checks for encoding and thought it required UTF8. But compare this server with the one you have working if on same hosting provider systems.
I saw your response to Rick - me thinks 2 Gig memory is pretty low. Would love to watch a realtime 'top' when you are running something massive like a restore.
A backup of a course should insert a thing to have the restore process read the CFG's URL parameter when restoring for full URL's to local server resources. That's why your build menu in Section 0 does work on mine. Those sorts of customizations were designed to get around the scroll of death. Your course is set to show one section at a time ... provides links to previous section and next section automagically in 2.9.
There were several items I 'doctored' by editing files.xml ... once I saw a pattern I did a global search/replace in nano to fix them. Some had up to 20+ references in the .xml file. All the references in the files.xml filenames that had () characters in their name were changed to remove those characters.
When the backup was extracted during the restore processes review screen, several references like this one:
[ [ pluginname ] ] Scientific Notation and Sig Figs Checklist
The [ [ pluginname ] ] means my server didn't have some addon you have installed on your 2.7. Have it on the 2.9? Is it compatible with 2.9? The name you gave it suggest 'Checklist' add-on. The restore did work even though I saw no 'checklist' resources in the course that restored.
So ... you could copy the code in that Section 0 and paste it into a TextEdit (know you use a Mac) file edit/store locally. You need to download the images in that Section 0 as well. Keep all of that in an archive folder should you want to put it back. Then remove it. When I view your course, it's 'hefty' processing just to get Section 0 generated and shipped to browser (am on a farily fast connection at home too!).
As per question in a private message sent to you, did you check out the quizzes that had images missing?
There are too many of them for me to check/find out. So please return to the site provided and check that those images are indeed there.
If everything looks ok in the course on my site, let me back it up (don't try the Moodle UI to backup). I do the command line thing out to a directory NOT in Moodle's filedir. I also 'watch' the build in realtime as well as logs while the backup is being run. Once I've backed it up sucessfully, I'll put it out somewhere for you to download. I've given you my EMail address. Please use it. And, please use your own EMail so I can communicate directly with you providing the download link. Once you have downloaded and communicated back that you have successfully downloaded, I need to remove any/all files, etc. related to the course.
BTW, I cannot promise that you will be able to restore this course to your system, but it's work a shot.
'spirit of sharing'. Ken
BTW, didn't say this earlier ... have to compliment you on that course. It's pleasing to the eye and I can tell you've put a lot of time into it. It's NO wonder you want to keep it and restore to your 2.9. ;)
Let me hear from you via regular EMail. Can't leave your course on the server where it resides right now.
'spirit of sharing', Ken