This is the first time in a year of administering Moodle that I have had to post a new topic rather than finding the answer already here, so I hope you can help!
First of all, here's what I'm working with...
Moodle version is: 1.8.2+ (2007021520)
Active Courses: 315
Database Server: MySQL (5.0.26)
PHP Version: 5.2.0
Our cron runs at 30 minute intervals and normally only takes 5 seconds. However, when Moodle's backup runs the cron script goes from taking 5 seconds to around 5 hours, this causes a huge performance hit and the page load time sky-rockets to anywhere upto 30 seconds. The backups, which are scheduled for midnight, are frequently effecting morning performance and I regularly have to kill the process to get the page load time down for morning classes.
Yesterday I changed crontab to save the output to date stamped files so I can see what is going on. Most the files are around 70KB however the log with backing up is 30MB and has an awful lot of whitespace and blank lines in it. Also, while I was in the shell I looked at the mail for root and it was chock full of messages from the cron execution...
For one cron execution these two lines are repeated 1412 times:
PHP Notice: Undefined property: stdClass::$instances in /srv/www/htdocs/backup/backuplib.php on line 2223 PHP Warning: Invalid argument supplied for foreach() in /srv/www/htdocs/backup/backuplib.php on line 2224
With this line being repeted twice:
PHP Notice: Undefined variable: instanceid in /srv/www/htdocs/mod/wiki/backuplib.php on line 188
I have attached the cronlog and shell output to this post.
I am aware there is a performance boost between 1.8.2 and 1.8.3 but unfortunately a full upgrade is not an option at this time. This is because I have inherited Moodle from the previous guy in my role and since this version was implemented there has been numerous code tweaks and add-ons which haven't been fully documented which would be lost in the event of an upgrade. Sorry to be a pain, but our College seems to have a policy about being trailing edge; upgrades don't tend to go down well unless there is a critical security need for them.
Thank you for any help and advice you can give me. Moodle is a amazing product and I am proud to be administering it.
I.T Development Officer
User-driven backup and restore is still available as a means of exporting/importing courses.
If you expect that you will need to restore per-course backups, what you can do is setup a secondary moodle install where you restore the database and moodledata. From that secondary installation, run a manual "backup" of the course requested and... voila!
I picked up on this thread when I experienced the same problem. The advise to drop backups via cron and use DB/moodledata backups instead is very useful as are points about user-driving course backup/restore still being available and the use of a secondary install of moodle if a cron backup was required. These options are available to me so I guess I will use them.
In the interest of curiosity however...
- Why are large reams of white space (an <br />) chars output? Would this not be classed as a bug?
- What does the cron backup do that makes it so time consuming (I notice my CPU utilisation yo-yos between ~100% and ~1% every 50 seconds or so
- With a moodle environment of 3151 users and 260 courses (5Gb of moodledata) - is this deamed large? (we provide moodle for the Derby Adult Learning Service)
Thanks for your feedback,
In Linux the best way would be to open in console htop and iostat -mx 5 in second one.
2) Course backup execution time depends on many factors. The size of any particular course is a big one. Also the performance capabilities of your server do play a part. Our experience has been that "automatic course backups" are not super reliable, for a variety of reasons, and we usually do not recommend them. Doing a full database backup at the server level is actually a much more effective, consistent, and complete way to backup a Moodle site. Look into using mysqldump for this process.
mysqldump -Q -c -q -udbuser -pdbuserpassword -hmysqlhostname moodledb > moodle_database_backup.sql
3) Your site sounds "medium sized" based on my experience so far. 5GB moodledata is getting up there, though. Certainly on the large end of the spectrum, but I have seen much larger.
we have similar problems...We are using Moodle 1.9.5+ (Build: 20090527) and the backupdata comes to about 75GB.
Whereas previously automated nightly backups would complete, if after 5-6 hours, it will now hang until a cron unlock kicks them off after 8 hours. It will then hang again...this is problematic as the cron lock will engage again and the normal cron routines will not get run.
However I have just ran the cron manually and it completed ok!? Why would the automatic cron fail to run the backups properly, but a manually run cron complete the backups?
Kindest Regards Daran
For such a large number of users and courses 5 GB is modest, I would say.
BTW, do you count the Moodle own backups (the zip-Files) to those 5 GB?
I am not a Linux knowledgable so to do these daily backups for my MOODLE is chinese I just lost a whole month of user contents due to a dabase error or connecting strings how can I recover my students and courses contents?
I need to set up daily automatic backup for MOODLE components how do I do this since I havent found a step by step process described here..
We had probelms with long backup times - nearly 24 hours for a site with about 800 small courses. We do have a large number of student accounts on the system - 4000+. We found that changing the setting for 'Users' in the backup settings from the default value of 'All' to 'Users' cut backup times down to just over an hour!