Backup don't finish the job...

Backup don't finish the job...

by Éric Bugnet -
Number of replies: 17
Picture of Documentation writers Picture of Plugin developers Picture of Translators

Hi, all...

I've got a problem with backing up all the site or only course :

All is OK to step : "Copying zipped file", and then Moodle says me : "An error occurred while copying the zip file to the course directory

My directory is on Chmod 777, using Debian and Moodle 1.4.3

I don't see anything on this forum for this problem... Can someone help me ???? Please ...

Thank's a lot, Eric

Average of ratings: -
In reply to Éric Bugnet

Re: Backup don't finish the job...

by Nina Hämäläinen -

Hi, Eric

I had a same problem. (Fedora 3, mysql-3.23.58-14, php-4.3.10-3.2, moodle 1.4.3 (latest). I couldn't make any automated or manual backups. 


When I installed moodle, I told paths to my zip and unzip programs.

# whereis zip
zip: /usr/bin/zip

# whereis unzip
unzip: /usr/bin/unzip

I removed those paths from Moodle configuration, variables page. And now my backups works fine!

BTW Could some explain me why Moodle gives  An error occurred while copying the zip file to the course directory, If zip and unzip paths are given? I think, that Moodle can make zip files, but it can't copy file into course directory. Directories are owned by apache:apache 777.

Hope this help someone!

nina

In reply to Nina Hämäläinen

Re: Backup don't finish the job...

by Eloy Lafuente (stronk7) -
Picture of Core developers Picture of Documentation writers Picture of Moodle HQ Picture of Peer reviewers Picture of Plugin developers Picture of Testers
Hi Tina,

that sounds really strange. Both zip alternatives (Moodle's built in and external zip commands) should create the same backup_xxxx_123345.zip file.

I cannot imagine why files zipped with your external zip doesn't work. Can you set the zip variable to such external commands, try a manual backup and then take a look under moodledata/temp/backup/123456789 (the recent one dir).

Inside it you should see some files like moodle.xml, an dirs like moddata, course_files.... Is there a moodle.zip file? What about its perms?

Ciao smile
In reply to Eloy Lafuente (stronk7)

Re: Backup don't finish the job...

by Nina Hämäläinen -
Hi Eloy!

OK,

/usr/bin/zip and /usr/bin/unzip variables are back in moodle configuration

Tried to backup Course HTML:

Course backup: HTML (HTML)

Name:
    backup-html-20050312-0819.zip

    * Creating temporary structures
    * Deleting old data
    * Creating XML file
          o Writing header
          o Writing general info
          o Writing course data
                + Course info
                + Sections
          o Writing users info
          o Writing categories and questions
          o Writing scales info
          o Writing groups info
          o Writing events info
          o Writing modules info
                + Assignments
                + Chats
                + Choices
                + Forums
                + Glossaries
                + Journals
                + Labels
                + Lessons
                + Quizzes
                + Resources
                + Scorms
                + Surveys
                + Wikis
                + Workshops
    * Copying user files
    * Copying course files
    * Zipping backup
    * Copying zip file

      An error occurred while copying the zip file to the course directory


The backup did not complete successfully

Continue

----------

Inside dir /moodledata/temp/backup/1110608384    
# ls -l
total 64
drwxrwxrwx  3 apache apache  4096 Mar 12 08:20 course_files
drwxrwxrwx  2 apache apache  4096 Mar 12 08:20 moddata
-rw-rw-rw-  1 apache apache 34912 Mar 12 08:20 moodle.xml
drwxrwxrwx  5 apache apache  4096 Mar 12 08:20 user_files

No zip files. sad

Apache user seems to use shell /sbin/nologin. I changed it
/bin/bash and tried to backup, but I got same result.

nina smile






In reply to Nina Hämäläinen

Re: Backup don't finish the job...

by Eloy Lafuente (stronk7) -
Picture of Core developers Picture of Documentation writers Picture of Moodle HQ Picture of Peer reviewers Picture of Plugin developers Picture of Testers
Hi Mina,

thanks for the exhaustive answer! big grin

It seems that your zip command has some problems zipping your files. Sometimes it's due to some "wrong" file name (diacritics...) or bad perms in some file/directory.

To check what is the zip command doing, we can try this (from the shell):
  1. Goto "/moodledata/temp/backup/1110608384" (or another problematic dir).
  2. Execute the zip command manually: "zip -r test.zip course_files moddata moodle.xml user_files".
  3. Examine the output. It should give us some answer to our problem.


Ciao smile

PS: If you want, we can introduce some minor modifications to Moodle code to be able to see how your zip command is being executed to trace it from there too (although it should be exactly like the command listed above).
In reply to Eloy Lafuente (stronk7)

Re: Backup don't finish the job...

by Nina Hämäläinen -
Thanks Eloy and Eric

My answer wasn't so perfect afterall -
I forget to mention that
zip command works manually ok (executed by root or apache)big grin

Let's try code modifications.

Nina


In reply to Nina Hämäläinen

Re: Backup don't finish the job...

by Eloy Lafuente (stronk7) -
Picture of Core developers Picture of Documentation writers Picture of Moodle HQ Picture of Peer reviewers Picture of Plugin developers Picture of Testers
Did it worked?!

Well, then we should check how Moodle is doing the work in your server. I suggest this:
  • Save a copy of your currently installed "lib/moodlelib.php" file (to be able to go back to the original).
  • Edit the original and make this changes:
    • Look for the zip_files() function.
    • Near de end of it (just before the start of the unzip_files() function) you will see this lines:
              //All converted to backslashes in WIN
      if (strtoupper(substr(PHP_OS, 0, 3)) === 'WIN') {
      $command = str_replace('/','',$command);
      }
    • Replace the last one (the Exec one) by these lines:
              echo '<pre>'.$command.'</pre>';
      $lastline = Exec($command,$output,$status);
      echo '<pre>'.$lastline.'</pre>';
      print_object($output);
      echo '<pre>'.$status.'</pre>';
    • Run the backup and you will see a lot of info near the end of the process.
    • Does such output throw any idea about your problem?


    Ciao smile
Average of ratings: Useful (1)
In reply to Eloy Lafuente (stronk7)

Re: Backup don't finish the job...

by Nina Hämäläinen -
Here is the result:

  • Zipping backup
    cd '/var/www/moodledata/temp/backup/1110725599' ;'/usr/bin/zip' -r '/var/www/moodledata/temp/backup/1110725599/backup-html-20050313-1653.zip' 'moddata' 'user_files' 'moodle.xml' 'course_files' 
    Array
    (
    )
    12
  • Copying zip file
            An error occurred....

Nina
In reply to Nina Hämäläinen

Re: Backup don't finish the job...

by Eloy Lafuente (stronk7) -
Picture of Core developers Picture of Documentation writers Picture of Moodle HQ Picture of Peer reviewers Picture of Plugin developers Picture of Testers
Hi Nina,

that's really strange because the command seems perfect but the output (the array) is completely empty and the status code of your command is "12" and looking for it in zip error codes it means "nothing to do"!!

Are you able to execute such command using the apache user? Is it possible that your path wasn't a real path but a soft-linked one and zip is working wrong or anything similar? Or some type of PHP's safemode enabled?

It's really strange, but it seems to be something related with your server and/or PHP installation. Sincerely, if you don't find anything wrong testing my prev-paragraph questions, I haven't more ideas about your problem! sad

Ciao smile
In reply to Eloy Lafuente (stronk7)

Re: Backup don't finish the job...

by Rosario Carcò -

Hi, I do not know whether this thread is still actual.

I have two systems running on different HW, with different settings, etc.

Moodle 1.5.2 works correctly

Moodle 1.5.2+ has the backup problem tracked here. (Error while copying...)

I issued the command:

www:/webserver_log/www.ourServer.ourCountry # tail -f error_log
sh: line 1: /cd: Datei oder Verzeichnis nicht gefunden

It seems the command '/cd' is either not executed correctly or the php-script tries to change into an inexistent directory. Is the slash really in the php-script? Moving that zip file should also be possible without cd I think.

Zipping from the command line as suggested works fine too. Permissions are ok too.

Rosario

In reply to Rosario Carcò

Re: Backup don't finish the job...

by Rosario Carcò -

It took me some weeks to track it down. Here my results:

1) On one system safe_mode is OFF and everything works fine, no matter whether using external utilities with /usr/bin/zip and /usr/bin/unzip or leaving those paths empty and thus using internal pclzip.lib.php functions

2) On the other system the responsible admin turned safe_mode=On in php.ini

After some tests I discovered that the slash in the previous post comes from the safe_mode_exec_dir setting when safe_mode is turned on. If I put a correct path in there, as should be, eg. safe_mode_exec_dir=/webserver/safe_mode_exec_dir, then the whole path is set before the cd command which is executed by the zip_files() function in our moodlelib.php, resulting in something like that:

/webserver/safe_mode_exec_dir/cd

which, again, does not exist. The admin had copied zip and unzip to that safe_mode_exec_dir path to allow the execution of those external utilities. But as I realised in my tests, the real problem is not COPYING the zip-file to its destination path, but the execution of the whole $command mentioned in the previous posts BECAUSE of safe_mode being on.

I think this explains, why the zip files are not there even if we succeed creating them manually as root or webserver users and why some run into this problem while others do not have it.

After that I dived into the problem of how to solve Moodle AND safe_mode. And the discussion is endless on php's side and on Moodle's side see the according forum:

http://moodle.org/mod/forum/discuss.php?d=1374#286772

If I find a viable solution to have safe_mode ON and do Moodle course backups, I will post it there.

We must differentiate also the case where Moodle is installed on a server where you have root access and where you DO NOT, eg. if you run Moodle on a provider's host.

In the first case, Moodle runs normally as the web-server-user, and apparently everything should work fine even with safe_mode turned on, which definitely is not the case, as we can see whith Moodle's course backups. Not because of the different UID/GID which are affected by the safe_mode but because the safe_mode_exec_dir alters the simple commands like cd. (Please correct me, if I am mistaken by my tests or this problem can be solved so that cd gets executed correctly in $command.)

On the other hand, on a provider's host, having safe_mode On, Moodle's php scripts run into problems much earlier because of the UID/GID being different when creating/copying/deleting directories and files.

I hope this explains why course backups are not possible with external utilities when safe_mode is On. Leaving the paths empty in that case, so that Moodle will use the built in pclzip.lib.php, will work correctly, if php was installed with zlib support.

Rosario

In reply to Nina Hämäläinen

Re: Backup don't finish the job...

by Éric Bugnet -
Picture of Documentation writers Picture of Plugin developers Picture of Translators

Hi, Nina...

Sorry, but I'm enjoy not be the only one with this problem !!! wink

Recently, I try to unzip a file in moodle (when you upload a zip file, you can unzip it in moodle), and it don't work too :

/bin/tar: essai.zip: Not found in archive
/bin/tar: /home/moodle/upload/5/Docs_ressources: Not found in archive
/bin/tar: Error exit delayed from previous errors

I will looking in that way, but I'm not the server manager, and I can't do many things alone sad

Edit :

I try to removed path : it's working fine no !

I don't understood !

Ciao

Eric

In reply to Éric Bugnet

Re: Backup don't finish the job...

by Eloy Lafuente (stronk7) -
Picture of Core developers Picture of Documentation writers Picture of Moodle HQ Picture of Peer reviewers Picture of Plugin developers Picture of Testers
Hi Eric,

but "/bin/tar" isn't a zip command but a different type of archive tool (not compatible with zip). You should use "/usr/bin/zip" in your configuration instead (or the location where the zip command is in your server).

Ciao smile
In reply to Eloy Lafuente (stronk7)

Re: Backup don't finish the job...

by Éric Bugnet -
Picture of Documentation writers Picture of Plugin developers Picture of Translators

I'm stupid ! timide

Thanks a lot, it 's working now...

Error was "Can't move file", so I looked in this direction !!!

Ciao !

In reply to Nina Hämäläinen

Re: Backup don't finish the job...

by Duarte Silvestre -

I haveexactly the same problem with Moodle 1.6.3 .

Did you found a solution?

In reply to Éric Bugnet

Re: Backup don't finish the job...

by Duarte Silvestre -

I felt this problem in the following Moodle versions:

- Moodle 1.7 beta

- Moodle 1.7+ latest branch (when upgraded from Moodle 1.7 beta)


Today I setup a fresh Moodle 1.7+ installation and the problem still continue.

Then, is not possible for me to do backups in Moodle 1.7 because, if I don't indicate the system paths do zip and unzip, I have this problem:

http://moodle.org/mod/forum/discuss.php?d=59765

My question: Backup in Moodle really works?

My System Paths:

Path to Zip: C:\UnxUtils\zip.exe
Path to UnZip: C:\UnxUtils\unzip.exe
Path to du: C:\UnxUtils\du.exe