Sorry for the length of this post, but I think I have found a solution for an error message I ran into when manually backing up courses.
The error message was "An error occurred while copying the zip file to the course directory"
It only started happening when I added "/usr/bin/zip" as the path for the zip command line utility. I really wanted to use the command line utility because it's faster than Moodle's internal zip library, and my school district is going to have several large Moodle courses on our system that we need to backup.
Filling in the field completely broke manual backups, but I noticed that automatic nightly backups were still working fine. So what's the difference? Well, manual backups are run by Apache, while automatic backups are performed by cron (I think cron runs as root, but, regardless, cron tasks are executed by a user with a shell).
So I tried messing with file permissions, with no success. I decided to try running the zip utility with the Apache user, but on my server the Apache user has no shell. This is for security reasons and is declared in /etc/passwd:
apache:x:48:48:Apache:/var/www:/sbin/nologin
I temporarily gave Apache a different shell (/bin/bash) so I could "su apache". After su-ing I found that I could run the zip command line utility.
Temporarily baffled, I started poking around with the Moodle source code, specifically moodlelib.php in the lib folder. I turned on a bunch of debug statements and found that this is the line executed when you specify the path to /usr/bin/zip:
Exec("cd '/var/www/moodle_data/staff/temp/backup/1198250478' ;'/usr/bin/zip' -r '/var/www/moodle_data/staff/temp/backup/1198250478/backup-rts-20071221-1021.zip' 'group_files' 'moddata' 'user_files' 'moodle.xml' 'course_files'");
This line of code should have created the zip file in a temporary location, but that just wasn't happening. So, I created a test PHP script where I could play around with this line of code. I discovered that if I specified the full path for each of the files I wanted to zip, a zip file would be created.
My conclusion is that Apache cannot run "cd" because it does not have a shell. And without changing directories, it cannot create a zip file because the source files (moodle.xml, etc.) can't be found.
Rather than give Apache a shell in the /etc/passwd, I think the more secure solution is to list each file by its full path.
This can be accomplished by tweaking a single line of code in moodlelib.php in the zip_files() function. Change:
$filestozip .= escapeshellarg(basename($filetozip));
To:
$filestozip .= escapeshellarg($filetozip);
The basename() function cut out the full path from the file list. I think this was done for the sake of clarity and tidiness, but the full path really is necessary if your apache user doesn't have a shell (which is how it should be).
Does anyone know how I can contribute this as an official patch?
Thanks!
Brandon
Possible fix for "An error occurred while copying the zip file to the course directory"
by Brandon Kish -
Number of replies: 6
In reply to Brandon Kish
Re: Possible fix for "An error occurred while copying the zip file to the course directory"
by Don Hinz -
Brandon,
Where is the moodlelib.php file located?
In reply to Brandon Kish
Re: Possible fix for "An error occurred while copying the zip file to the course directory"
by Timothy Takemoto -
Three years later, thank you so much Brandon! On my last server, apache was running as root. On the new one I have problems moving, cd-ing files.
But it seems to occur only with large files. I wonder if not having a shell (or not being root) might effect the size of the files that can be moved.
In any event, the above is very thorough and helpful. I will try this patch.
to those wondering where moodlelib.php is, it is in the folder moodle/lib/
In reply to Timothy Takemoto
Re: Possible fix for "An error occurred while copying the zip file to the course directory"
by Timothy Takemoto -
Alas it did not work for me.
As always I get the, "An error occurred while copying the zip file to the course directory The backup did not complete successfully"
It works just as well though
Thanks to your suggestion, I think that it is probably related to the difference in privelidges given to root and other users.
Hence root can do more to bigger files than other users, including apache.
Tim
(if anyone has any suggestions, please contact me via my homepage given on my profile)
In reply to Timothy Takemoto
Re: Possible fix for "An error occurred while copying the zip file to the course directory"
by Luis de Vasconcelos -
Timothy, are you on a Unix or a Windows machine?
I haven't been able to get Moodle to use an external .zip utility on any of my Windows 2003 boxes. I don't know if it also applies to Apache, but on IIS I've read that the IIS User needs to have execute access to cmd.exe. Allowing this does open up some possible security issues, but try it and see if it solves your problem (this is when a dev box is useful!)
Which external .zip utility are you using?
In reply to Luis de Vasconcelos
Re: Possible fix for "An error occurred while copying the zip file to the course directory"
by Timothy Takemoto -
I am using a unix machine. I am not sure of the flavour but I think that it may be using BSD Unix, and I am not sure of the type of zip routine.
By the way *STOP PRESS* I found out one reason for the occurence of this error.
The backup process stop(on old vesions of moodle) if you enter a filename which does not end in ".zip"! As simple as that.
But there are other things wrong with my backup and restore processes. It may be due to the nonstandard modules (TUI, attendance slip) that I am using. I will experiment without them.
Tim
In reply to Timothy Takemoto
Re: Possible fix for "An error occurred while copying the zip file to the course directory"
by Timothy Takemoto -
I also looked into "ulimit" This is a unix command to limit the size of the files that can be manipulated, opened, piped, stored by any particular user. The shell command "ulimit -a" will tell you what limits have been placed on your own user. I am not sure if that is the same as that of apache and php. Assuming it is, then there are no file size limits placed on my user.
I also find that one massive, 2GB course is unzippable. It seems to be course that are almost exactly 100MB that go wrong but perhaps that is a conicidence.
The same course can be unzipped on the old server which has apache running as root, and that has unzip installed. I think that one of these two things - that apache is root, and that unzip is installed is allowing the course to be unzipped and restored on the old server. T
This indicates to me that the problem is not in the modules that I am using but it really is the unzipping that is the problem.
It could be the number of files in the course. There are about 400 in the course backup that I am attempting to unzip. But I am allowed 5000 processes and 11000 open files. Perhaps for some reason the unzipping reqires 10 processes per file! But if it were the case that php uses a lot of processes in the unzipping then it would explain why it does not work on the new server. root, using the unzip routine may use fewer processes, and be allowed more anyway.
But really I am in the dark. I think that even if my sysop were to install unzip, then there would be a 50 50 chance that I would still not be able to restore.
T