!!! error/error_zip_packing !!!

!!! error/error_zip_packing !!!

by Pedro Ximenes -
Number of replies: 12

Hi Moodlers,


Environment: Moodle 2.8.2+, Oracle Linux 6.6 x64, Kernel 3.


I have some courses that have more than 4GB and have automated backup. These courses can not complete the backup.


After a few searches, I found the problem: the ZIP command can not be completed in courses with more of 4GB.


So, I did a test: tried mannualy backup a course that have 20GB using the command "php admin/cli/backup.php --courseid=19082 --destination=/backup/moodle/".


The error was:

== Performing backup... ==

!!! error/error_zip_packing !!!


A new test was made: I renamed the file /usr/bin/zip and created a link to /bin/gzip for /usr/bin/zip. And the result was the same.


We can not reduce the size of the courses, but we need the backups.


Can anyone help? Suggestions? Use of new feature "enabletgzbackups" is OK?


Thanks a lot!


Pedro Ximenes

Average of ratings: -
In reply to Pedro Ximenes

Re: !!! error/error_zip_packing !!!

by Ken Task -
Picture of Particularly helpful Moodlers

Don't run the same OS, but have had the same issues and sharing what was discovered.   The error isn't accurate per say ... but related.

The backup routine creates a hash named directory in moodledata/temp/backup/ that it uses for building the various pieces to a course backup.   Directory name looks something like 23asoqwerudp4235.

A large course will take time so one could watch the process to see if it indeed did complete but could NOT 'cp' the backup to it's desitnation ... which is what my issue was (use of cp rather than mv).

So check out your moodledata/temp/backup/ directory for those long hash named directories.

If you have one there, change into it and see if there is not already residing there a large .mbz file.   If there is, one can 'mv' to some other location:

mv backup-moodle2-course-6-ma101-20150129-1623.mbz /home/backup/

It is safe to remove manually those hash named directories on failed backups.

Try your command line backup like so: php backup.php --courseid=6 --destination=./

Then ls *.mbz if it completes.   One could then mv *.mbz /whereever/

Another item to check/verify, max_input_vars setting of your DB server.   MySQL has a default lower than the number needed to do large backups.

Granted this is not a solution, but something to investigate/try ... you said you needed those backups! ;)

'spirit of sharing', Ken

Average of ratings: -
In reply to Ken Task

Re: !!! error/error_zip_packing !!!

by John Jackson -

Ken,

Funny I bumped in to you here as well. smile

I too am getting the zip packing error. My environment is LAMP, CentOS specifically. 2.8.3+

The automated backup fails. When I go into a course and run a manual course backup I get the zip packing error.

When I look in the temp/backup directory I find that Moodle had copied a bunch of the course files over to temp, but did not make any zip file, and the log file in temp/backup is 0 KB in size, so there's not much to go on.

Where would I find that setting in MySQL for max_input_vars, and what would be a good number to put in to manage large course backups?

thanks!

John

Average of ratings: -
In reply to John Jackson

Re: !!! error/error_zip_packing !!!

by Ken Task -
Picture of Particularly helpful Moodlers

In a version 2 of Moodle, zip is used but the file it creates has a .mbz extension. (which stands for 'moodle backup zip '.   In the hashnamed directory where a backup is being built, is there an .mbz file there?   What I've found is the last part of the automated backup process IF set to save in 'course area' ... which is really filedir under the new file system ... uses a copy command and therein lies the issue.   copy command cannot be used with files 40Gig+.   If one finds a .mbz file there, Moodle did complete the backup and the .mbz file IS probably a good backup but to archive it some where one needs to use command line 'mv' and can only 'mv' to a directory outside of Moodle's file system.

Once having moved the .mbz to another location, one can remove the 'hashnamed' directory being used for building the backup.

The question about max_input_vars indicates you've seen some error in a log.   Yes?   What did that error say.

The default for that is 1000.  Can't give you a specific value cause it's kinda tied to your issue with large courses I would imagine.   So ... increase it to say 2000.  Enter a line in php.ini

max_input_vars = 2000;

to increase it and restart apache for it to take affect.

On a related note, I've seen posting today that using 'experimental' tar.gz for backups has helped some folks past the issue you are having.    Might try that.

'spirit of sharing', Ken


Average of ratings: -
In reply to Ken Task

Re: !!! error/error_zip_packing !!!

by Ken Task -
Picture of Particularly helpful Moodlers

Correction to posting ... limit is NOT "40Gig+" but 4Gig - which is easy to reach with  courses involving uploaded audio/video.

Ken

Average of ratings: -
In reply to Ken Task

Re: !!! error/error_zip_packing !!!

by Tim Hunt -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

See this other thread that is going on now: https://moodle.org/mod/forum/discuss.php?d=282298. You should turn on that 'experimental' settings.

Average of ratings: -
In reply to Tim Hunt

Re: !!! error/error_zip_packing !!!

by John Jackson -

After I enabled the "experimental" compression, I found I can manually backup courses now that result in a zipped file larger than 4 GB, which is a plus. I did the manual backup from within a course, using the "backup" option under the course management menu.

However, the automated course backup run by cron still fails with the same course than I can manually backup without error.

Are there logs somewhere that would give more information than just "failed"?

Any other ideas folks have out there?

Thanks in advance....

John

Average of ratings: -
In reply to John Jackson

Re: !!! error/error_zip_packing !!!

by Ken Task -
Picture of Particularly helpful Moodlers

Is there anything left in moodledata/temp/backup/ after an automated run?

Larger than 0 byte size .log files might give a hint/clue.

Do you have automated backups set to the alternative directory (outside of moodledata/filedir)?

By chance are there backups contained in the backup?

Think I'd try:

setting autobackups to manual

setting the save directory to be outside of Moodle ... where ever you have enough room/space - ONLY .. not in both locations.   Example: /home/mbackups/

Run the auto backup via command line to test:

cd moodlecode/admin/cli/automated_backups.php

* before doing that, however, would open up 2 more terminal sessions to the server as root user

1. cd pathto/moodledata/temp/backup

and in that terminal session run: watch "ls -lR"

2. cd pathtoaltdirectory for backups (let's say you set up /home/mbackups/

in that terminal session: watch "ls -lR;df"

In the #1 window above one can see the course backups being built.

In the #2 window above one can see IF the built backup reaches it's destination.

Lot's of trouble, I know, but other than just running scripts (with/without debug) it's a way to actually see what's going on and at which stage it fails.

'spirit of sharing', Ken



Average of ratings: -
In reply to Ken Task

Re: !!! error/error_zip_packing !!!

by Jinxing Chen -
I got same error as well.

"So check out your moodledata/temp/backup/ directory for those long hash named directories.

If you have one there, change into it and see if there is not already residing there a large .mbz file.   If there is, one can 'mv' to some other location:"

What about I can only found long hash name on /moodledata/temp/backup/ directory, but there are no .mbz file to 'mv'.

Please help.
Average of ratings: -
In reply to Jinxing Chen

Re: !!! error/error_zip_packing !!!

by Ken Task -
Picture of Particularly helpful Moodlers

@Jinxing Chen

Change into 'long hash name' directory.   Then ls -l ... list long.  If there is a backup.mbz file there, it's more than likely a valid backup that couldn't be 'copied' to destination.  Use 'mv' on the backup.mbz to any location outside moodledata or to a file system repository in moodledata.

If you see a backup.mbz.gasee23 kinda file name, that's a temp file that was being built and it is NOT a valid/completed backup.

However, if one sees a moodle_backup.xml file there, along with other .xml files and corresponding directories, the building of the mbz could have failed.   But, you could try removing the temp backup.mbz.qffwer file, then making a .mbz of your own:

tar -cvzf mybackup.mbz ./*

while in the 'long content hash' directory.

Not a 100% deal, but might be worth a shot!

Check PHP settings for time for a script to run and other such settings ... increase them.  You could also turn on debugging, do a backup process, to see what debugging might offer.   Definitely look at apache error logs or php error logs.

The only way to know if mybackup.mbz will work or not is to try to restore it and after the first couple of steps in restoring, cancel the operation ... assume that it is good. (fingers crossed)

'spirit of sharing', Ken


Average of ratings: -
In reply to Pedro Ximenes

Re: !!! error/error_zip_packing !!!

by Pedro Ximenes -

Hi there!


Thanks all for your replies.


I've been enabled GZIP compression, so now 99% of courses are being backed up. Now, only front page have backup problems. The reason, I guess, is because we say to backup logs and didn't delete users.


Whatever, the most important is OK for now!


Regards,


Pedro

Average of ratings: -
In reply to Pedro Ximenes

Re: !!! error/error_zip_packing !!!

by Emma Richardson -
Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Plugin developers
You cannot restore the front page anyway so not really sure why that backups at all.  Unless something has changed in the latest releases...
Average of ratings: -