Backups are not completed

Backups are not completed

Mike Prophete -
回帖数：19

I am having problems with backing up moodle courses. I tested backups settings for manual and automated backups by setting a cron job to process the backups. I  notice that  no backups found under the course area backup section. I have viewed the backup reports under site administration->reports->backups and it listed successfull backups but my backups are no where to be found. I aslo change the file path from course area to a specific directory and ran the backups but the backups wre not found. I have checked the php errors logs and found that all of my backups of failing. The php is error is: "backup_auto_failed_on_course 'Course Name'".

I having this problem with multiple moodle servers running different version. My production server is running 2.1, my moodle test server is running 2.1, and my moodle development server is running 2.1.1. All three servers are having this backup issue.

I was only able to backup the production server once since the backups were activated. The moodle test server was backing up fine until I disable it backups for testing purposes against the moodle production server.

Thanks Mike

平均分： -
回复Mike Prophete

Re: Backups are not completed

Ken Task -
Particularly helpful Moodlers的头像

Truth in assisting ... don't have the same problem, but have been exploring the file system of a 2.1.1 recently so none of the following may lead anywhere, but ....

I know this will sound strange, but ... get into a course and in the course Admin menu, click 'Restore'.  What do you see?

You should see 'Course backup area', 'User private backup area', and 'Automated backups' (if that option turned on).

When choosing the Automated backups AND selecting another directory does that directory exist (it won't create it), permissions on that directory, is it located outside of Moodle web root?

In the datadirectory there is a temp directory.  Contained therein a 'backup' directory.  Think  that successful backups one will see something like:

-rw-rw-rw- 1 apache apache 0 Sep 27 13:03 797031ef9142f38b71ca12fbbfd07173.log

That's 0 byte size so nothing in them.  If your's are larger than 0 bytes, view one of them ... might give a hint/clue.

'spirit of sharing' (and exploring), Ken

 

平均分： -
回复Ken Task

Re: Backups are not completed

Mike Prophete -

Thanks for replying Ken,

I should have mention my moodle setup, I am running on Win 2K8R2, IIS7.5, & MSSQL 2005. I understand what you are saying about permission on the file directory but I change the backups to go to both places. Under the course administation-> restore, I do not see any backups listed in there. Whats odd that the production sever did its first backup within the same file directory that it wont backup to now . I have shared the directory to everyone with full rights on the backup folder and still the same errors are found in the php error log file.

I did find the log files in the moodledata directory for the backups found under the moodledata->temp->backup folder. Each course had a log file that says

"[Time Stamp] [warn] backup_auto_failed_on_course_name"

平均分： -
回复Mike Prophete

Re: Backups are not completed

Ken Task -
Particularly helpful Moodlers的头像

What's reported with Site Admin -> Development -> Debugging turned on when you run a backup manually on one course?

Web server logs?

平均分： -
回复Ken Task

Re: Backups are not completed

Doug Moody -

Ken,

I too have been exploring the 2.1.1 directory structure.

However, there are several folders missing that no one has been able to explain to me.

For example, I have no moodledata directory.

You mentioned "datadirectory" I have no such animal. However, my backups DO work. I wonder where the data isbeing stored? So far, I have not discovered where it's going. SOmewhere, obviously, but it sure isn't obvious!

平均分： -
回复Doug Moody

Re: Backups are not completed

Ken Task -
Particularly helpful Moodlers的头像

In your Moodle folder - the one accessible by the web - there is a config.php file.  It contains the path/location to the moodledata folder (term 'moodledata' used in a generic sense to indicate whatever folder is used for the Moodle).

'spirit of sharing', Ken

平均分： -
回复Ken Task

Re: Backups are not completed

Doug Moody -

Ken,

My site has a folder called "Moodle" at the root of the host. So, my website is http://xxxxx.com/moodle

Within the Moodle folder are of course all my other Moodle support folders, such as admin.

I thought that when people were saying "moodledata folder" they were referring to a folder actually called "moodledata" So you are saying there isn't really a folder with this name, right?

If that's the case, then what folder IS used for backup data? I use 2.1.1.

平均分： -
回复Doug Moody

Re: Backups are not completed

Ken Task -
Particularly helpful Moodlers的头像

A 'typical' (default) install of Moodle places the data folder outside of web root - ie, your http://site/moodle in a folder above webroot.

On a default CentOS server webroot is defined by apache to be /var/www/

The web pages servered out are in /var/www/html/

Many place the 'moodledata' folder in /var/www/

There is a data folder of some kind ... some where ... it's just not named 'moodledata'.  Open the config.php file located in your moodle folder.

It has config statements in it like:

$CFG->wwwroot   = 'http://www.site/moodle';

$CFG->dirroot   = '/var/www/html/moodle'; (where the moodle app lives)

$CFG->dataroot  = '/var/www/moodledata'; (** this is the data folder)

The 'moodledata' folder in the example above could be called just 'data' or whatever the installer script called it when installed or whatever the human that installed it called it.

In Moodle.org forums the term 'moodledata' refers to the default install of Moodle - but it does NOT mean that's the name of your data folder.

Check your config.php file, please.

'spirit of sharing', Ken

平均分：Useful (1)
回复Ken Task

Re: Backups are not completed

Mike Prophete -

Alright I found a big problem, when I tried to create a single backup under the course administration-> backup, I received a 404 error. I noticed in the url it was calling http://moodle_site/backup/backup.php once I clicked the perform backup button. I went ahead and did some digging around in my \\moodle\backup folder to see if the file actually exist . The file does exist and I changed the permission to Everyone with full rights to see if its a permission problem. I still receiving a 404 error with permission is set to full rights to everyone. I thought maybe the file corrupted but I can open the file with a text editor without any problems. 

As far as the logs files, I do see anything in the webserver log that specify anything related to backups. The only log file the mentions a backup problem is the php error log that I posted earlier in this thread.

平均分： -
回复Mike Prophete

Re: Backups are not completed

Ken Task -
Particularly helpful Moodlers的头像

Is "\\moodle\backup" a typo? By that I mean, on windows \\something indicates a network share, doesn't it?  If that's true, then apache is reporting correctly (404).  The protocol used by Moodle is http:// not a protocol normally used to connect to a shared folder.

While there are multiple ways to set up a backup directory outside of Moodle (could be a Windows server to which one has a mounted on the Moodle server), Moodle is using http - no smb:// or other file system protocol.  One could setup a folder at c:\ for a folder called "moodlebackup" and in Apache config have that folder accessible by Moodle to be able to save backups, but, if one does that, protect that folder so it's accessble only by the Moodle server and not directly by anyone using a browser.

Hope that helps.  'spirit of sharing', Ken

平均分： -
回复Ken Task

Re: Backups are not completed

Mike Prophete -

Yes that was a typo. I understand to what you are saying but the backups should have went into the coursearea to my understanding is a predefined location or not? I received a 404 https://moodlesite/backup/backup.php. I can access the other php files from the moodle site without any problems.The occurs at the last step to click on the "Perform Backup" backukp. Do you know what is the backup.php file is calling that would cause this issue.?

平均分： -
回复Mike Prophete

Re: Backups are not completed

Ken Task -
Particularly helpful Moodlers的头像

In version 2 of Moodle, there is only a "filedir" folder (no more course ID# folder in 'moodledata' [generic term]).  All files uploaded or created by Moodle (course backups) are hashed, the filename extension is stripped off, and the filename changed to what ever the hashing routine does.

If one sets up Automated backup to save the .mbz file to an alternate location as well (say a /home/backups/moodle2/ folder) then the backup is not only hashed in the new file system, but also in /home/backups/moodle2/ with it's full name and extension (.mbz).

Hoping this helps!

'spirit of sharing', Ken

平均分： -
回复Ken Task

Re: Backups are not completed

Mike Prophete -

Ken, thanks for explaning how are the backups are stored in the moodle "filedir" folder but lets us back track a little. At this point I can not create a single course backup for whatever reason now. As you mention above about testing what debugs on the page during a single course backup. I turned on debugging and this is what I am getting for a single course backup.

[05-Oct-2011 09:43:33] instantiating backup controller 503729e2c24a1b50cb58f06b3cbd89a5
[05-Oct-2011 09:43:33] setting controller status to 100
[05-Oct-2011 09:43:33] loading controller plan
[05-Oct-2011 09:43:33] setting controller status to 300
[05-Oct-2011 09:43:33] applying plan defaults
[05-Oct-2011 09:43:33] setting controller status to 400
[05-Oct-2011 09:43:33] checking plan security
[05-Oct-2011 09:43:33] setting controller status to 500
[05-Oct-2011 09:43:33] checking plan security
[05-Oct-2011 09:43:33] checking plan security
[05-Oct-2011 09:43:33] checking plan security
[05-Oct-2011 09:43:33] saving controller to db
[05-Oct-2011 09:43:33] calculating controller checksum 62d8b60c398f0198ceeb9f6659d5cb8e
[05-Oct-2011 09:43:37] loading controller from db
[05-Oct-2011 09:43:37] checking plan security
[05-Oct-2011 09:43:37] checking plan security
[05-Oct-2011 09:43:37] checking plan security
[05-Oct-2011 09:43:37] saving controller to db
[05-Oct-2011 09:43:37] calculating controller checksum 62d8b60c398f0198ceeb9f6659d5cb8e
[05-Oct-2011 09:43:41] loading controller from db
[05-Oct-2011 09:43:41] checking plan security
[05-Oct-2011 09:43:41] checking plan security
[05-Oct-2011 09:43:41] checking plan security
[05-Oct-2011 09:43:41] saving controller to db
[05-Oct-2011 09:43:41] calculating controller checksum 62d8b60c398f0198ceeb9f6659d5cb8e
[05-Oct-2011 09:43:43] loading controller from db
[05-Oct-2011 09:43:43] checking plan security
[05-Oct-2011 09:43:44] checking plan security
[05-Oct-2011 09:43:44] setting controller status to 700
[05-Oct-2011 09:43:44] saving controller to db
[05-Oct-2011 09:43:44] calculating controller checksum e4b752dca9bc3dfb4ee150d14b4f8c53
[05-Oct-2011 09:43:44] loading controller from db
[05-Oct-2011 09:43:44] setting controller status to 800
[05-Oct-2011 09:43:44] PHP Warning:  unlink(D:\moodledata/temp/backup/a0e6fcf100e53df82d4d03cb1c1bebec/users.xml): Permission denied in D:\moodle\backup\util\helper\backup_helper.class.php on line 103
[05-Oct-2011 09:43:44] Default exception handler: error/cannot_empty_backup_temp_dir Debug:
* line 48 of \backup\util\helper\backup_helper.class.php: backup_helper_exception thrown
* line 58 of \backup\util\helper\backup_helper.class.php: call to backup_helper::clear_backup_dir()
* line 143 of \backup\util\helper\backup_helper.class.php: call to backup_helper::delete_backup_dir()
* line 38 of \backup\moodle2\backup_stepslib.php: call to backup_helper::delete_old_backup_dirs()
* line 34 of \backup\util\plan\backup_execution_step.class.php: call to create_and_clean_temp_stuff->define_execution()
* line 153 of \backup\util\plan\base_task.class.php: call to backup_execution_step->execute()
* line 148 of \backup\util\plan\base_plan.class.php: call to base_task->execute()
* line 105 of \backup\util\plan\backup_plan.class.php: call to base_plan->execute()
* line 296 of \backup\controller\backup_controller.class.php: call to backup_plan->execute()
* line 111 of \backup\util\ui\backup_ui.class.php: call to backup_controller->execute_plan()
* line 86 of \backup\backup.php: call to backup_ui->execute()

 

I noticed that I received a permission denied on that file but why? No permission were change that directory.

平均分： -
回复Mike Prophete

Re: Backups are not completed

Ken Task -
Particularly helpful Moodlers的头像

We both can see:

[05-Oct-2011 09:43:44] PHP Warning:  unlink(D:\moodledata/temp/backup/a0e6fcf100e53df82d4d03cb1c1bebec/users.xml): Permission denied in D:\moodle\backup\util\helper\backup_helper.class.php on line 103

is the issue, I think.

What is D:\ - a secondary HD on the same server or a share mapped to D: on Moodle?  Aren't Windows permissions like Linux in that they are heirarchal? (top down, not bottom up).

Permission denied is the key.

Remember it's Moodle running under Apache (which has a user/group - or should have) that's attempting to write to D:\

Check permissions on D:\ then D:\moodle then D:\moodle\backup

moodledata/temp too!

If anyone reading this runs Windows, maybe you'd like to 'jump in' - am a Linux person myself! ;)

'spirit of sharing', Ken

平均分： -
回复Mike Prophete

Re: Backups are not completed

Ken Task -
Particularly helpful Moodlers的头像

Ok, after surfing the net looking for info on Windows and unlink, the 'unlink ... blah, blah, blah' error has been reported as a permissions issue in many open source apps running PHP on Windows.

As I re-read the error:

[05-Oct-2011 09:43:44] PHP Warning:  unlink(D:\moodledata/temp/backup/a0e6fcf100e53df82d4d03cb1c1bebec/users.xml): Permission denied in D:\moodle\backup\util\helper\backup_helper.class.php on line 103

Your moodledata folder as defined in the config.php for your Moodle appears to be: D:\moodledata\ That correct?  IS there a temp folder inside there and does it have the same permissions (read/write/execute) as D:\moodledata?  IF temp doesn't exist, create one and set permissions.

The backup routine collects info from Moodle and then uses the temp folder to 'construct' a backup .mbz file.  It must not only create an .xml file for a variety of things ... one being users (see error on writing/creating users.xml above)  but it must be able to create other directories and .xml files to build the .mbz file.

At this point it makes no difference really what changed and how that happened, let's concentrate on getting it to work.  Invesitgate the why later!

'spirit of sharing', Ken

平均分： -
回复Ken Task

Re: Backups are not completed

Mike Prophete -

I changed the permission on that directory and ran a manual backup by calling the backups through the moodle cli in the command prompt. The backups begins but gets stuck on first course. As of right now that course has been backing up for 2hrs. I checked the php errors and I only receive this error several times during the course backup.

[12-Oct-2011 15:09:49] Default exception handler: Sorry, the requested file could not be found Debug:
* line 429 of \lib\setuplib.php: moodle_exception thrown
* line 1542 of \lib\filelib.php: call to print_error()
* line 708 of \pluginfile.php: call to send_file_not_found()

平均分： -
回复Mike Prophete

Re: Backups are not completed

Ken Task -
Particularly helpful Moodlers的头像

Is there a 'temp/backup' directory in the data folder for your Moodle?

From observations and poking and probing with a Moodle that had issues with backing up a single course, found that folder is used as a temporary 'construction area' ... first file it writes is an XML file which contains all the info for a portion of the backup.  In one of your postings about this, the first error is about creating that xml file.  If it can't do that, the rest of the routine will fail.

Not sure you've responded about D:\ IS ... IS it a physical drive on the same server? or is it a drive that is mapped via Windows protocols? (the folder actually resides on another physical server).

Reason I ask, Moodle will use internet protocols to write to it.  IF a mapped drive and a true Windows share, Moodle may not be allowed by Windows to write to it via the methods Moodle is using.

On a Linux server I can mount a Windows share and have it appear to Apache as a local folder that has been configured to allow Apache/Moodle to write to it.  Realize that doesn't help you, just attempting to help 'talk' you through trouble shooting.  Maybe something I say/suggest will help you stumble upon the solution.

Let's try this (automated backups set to execute in Moodle):

cd [moodlewebroot]/admin/cli/

Then issue the following command:

php automated_backups.php

For each course in the Moodle, a hashed numbered folder will be created in [moodledata]/temp/backup/

As each course is backed up the following will be created:

./946957ddb621768cd984dfc495c8da59:
total 140
drwxrwxrwx 19 root root  4096 Oct 12 17:01 activities
-rw-rw-rw-  1 root root    79 Oct 12 17:01 completion.xml
drwxrwxrwx  3 root root  4096 Oct 12 17:01 course
drwxrwxrwx 12 root root  4096 Oct 12 17:02 files
-rw-rw-rw-  1 root root  8209 Oct 12 17:02 files.xml
-rw-rw-rw-  1 root root  1762 Oct 12 17:01 gradebook.xml

and the temporary .mbz will be created.

Here's what it looks like:

[root@moodle cli]# php automated_backups.php
Server Time: Wed, 12 Oct 2011 16:56:23 -0500
Checking automated backup status...OK
Getting admin info
Checking courses
Skipping deleted courses...0 courses
Running required automated backups...
Backing up Kens Moodle 2...complete - next execution: Sunday, 16 October 2011, 12:30 AM
Backing up Sys Admin...complete - next execution: Sunday, 16 October 2011, 12:30 AM
Backing up Build Your Own LAMP Box - short course (byolb)...complete - next execution: Sunday, 16 October 2011, 12:30 AM
Backing up Podcasting 101...complete - next execution: Sunday, 16 October 2011, 12:30 AM
Backing up Moodle Networks and Networking (mnet)...complete - next execution: Sunday, 16 October 2011, 12:30 AM
Backing up Installing a SandBox Moodle for Training...complete - next execution: Sunday, 16 October 2011, 12:30 AM
Backing up Basic Moodle Administration (basic)...complete - next execution: Sunday, 16 October 2011, 12:30 AM
Backing up Advanced Moodle Administration (golinux)...complete - next execution: Sunday, 16 October 2011, 12:30 AM
Backing up SandBox (sandbox)...complete - next execution: Sunday, 16 October 2011, 12:30 AM
Backing up Android Apps...complete - next execution: Sunday, 16 October 2011, 12:30 AM
Backing up Exploration into VirtualBox (exploring)...complete - next execution: Sunday, 16 October 2011, 12:30 AM
Sending email to admin
Automated backups complete.
Automated cron backups completed correctly
Execution took 462.815278 seconds

NOTE the execution time ... might need to increase your PHP settings IF backups of the courses are large.

Dunno if Windows has such a thing as the 'watch' command, but on a Linux server one can change into the working directory for backups and watch them happen with:

watch "ls -l"

while located in [moodledata]/temp/backup/

Useful if troubleshooting ... like we are right now! ;)

'spirit of sharing', Ken

平均分： -
回复Ken Task

Re: Backups are not completed

Mike Prophete -

Hey Ken,

The file path you see if the logs is not a network share. Its a secondary drive volume that I setup moodle on. I wanted to separate the moodle files from the windows file due our backukp program.  The backukp program will only back volumes not individual folders. I can backup only moodles files  without backing up the windows  and other files that resides on the common C:\ drive. 

Back on track, I have ran the automated_backup.php through the cli and received this error in the logs

[13-Oct-2011 09:01:32] PHP Warning:  unlink(D:\moodledata/temp/backup/474ef9936970817d0124e4bd0883ab9a.log): Permission denied in D:\moodle\backup\util\helper\backup_helper.class.php on line 146.

 

Again I given every user permissions on the "D:\" with full rights. I went ahead and found the service that moodle is using an a web application in IIS7.5 and gave it full rights on the server to try to cure this "Permission Denied".

For the php settings if you are referring to the amount of memory php can use its set to 512MB. My largest course is about 800MB. So should I changed the php memory alloaction to match my largest course or is 512MB is enough?

Thanks,

Mike

平均分： -
回复Mike Prophete

Re: Backups are not completed

Ken Task -
Particularly helpful Moodlers的头像

Am not a coder nor do I run Windows, so here's what I've been able to 'dig out' from the Web:

http://php.net/manual/en/function.unlink.php

and some copied comments from the discussion at the above site:

 
kgy 25-Nov-2010 10:34
a note to the windows unlink problem. I had some code running on a linux server, unlinking files fine, but when i switched to my home computer running on windows, i suddenly had the unlink - permission denied problem. After trying everything (all those things with ownership, etc.), i found out that my file (a temporary file created with tempnam) was not closed, therefore it couldn't be unlinked. I hope this helps someone outthere 

lpatrick 25-Oct-2010 09:08 unlink can fail after using ODBC commands on the file (on Windows).

Neither <?php odbc_free_result($result); ?> nor <?php odbc_close($conn); unset($conn); ?> did the trick.

Only <?php odbc_close_all(); ?> released the file such that it could be deleted afterwards ..

Apologize for the mess above ... can't clean up the junk! 伤心

You might have discovered a bug ... but for Windows only as I've not had the issue on Ubuntu nor CentOS servers.  Have done all I can do for ya, Mike.  Afraid I'll have to 'bump' this one to someone more knowledgeable than I in the Windows platform.  ANYONE a Windows Ace that can help Mike?

Best of luck.

'spirit of sharing', Ken

平均分： -