Course back-up process failing/timing out

Re: Course back-up process failing/timing out

by An sree -
Number of replies: 19

Hi,

Thanks for your reply, 


Actually this happened to a particular course, not for allThis is very large sized course, Is this affect the backup process?? . The site times out very fast and lands me at the login page if we refresh the backup page before execution. What you think??

In reply to An sree

Re: Course back-up process failing/timing out

by Ken Task -
Picture of Particularly helpful Moodlers

Don't think refresh will work - it kicks you out of backup process and back to front page .... right?   You might end up refreshing the front page ... server might still be faster than you! ;)

Again .... you have access to the course still.   Go into files, backups to see if there are multiple backups there.

Edit the course and either re-upload the images ...  errant references  ... Week 1/Lesson 1 etc. OR remove them from the course.

'spirit of sharing', Ken




In reply to Ken Task

Re: Course back-up process failing/timing out

by An sree -

Hi,

I got "A server error that affects your login session was detected" message when I print post variable in backup page. I think the session has timed out automatically for this course.What you think??


I couldn't face this issue when I am trying to backup other courses. This course also shows same missing asset messages while creating a backup. So i think  the missing images are not affecting the backup process.


Thanks in advance.

In reply to An sree

Re: Course back-up process failing/timing out

by Ken Task -
Picture of Particularly helpful Moodlers

Other courses are smaller then, right?

Check php settings for things like time for a script to run and memory a script can use.  Found on the Server -> PHP Info page in Moodle.   If time for a script to run is like 30 seconds, increase.   Any changes you make to php.ini do require the restart of apache.

Again:

you have access to the course still.   Go into files, backups to see if there are multiple backups there.

Remove multiple backup files.  You are attempting to reduce the size of the backup you are creating.

Check your apache error logs to get a clue.

Turn  on debugging in Moodle to see what it might report if anything.

'spirit of sharing', Ken

In reply to Ken Task

Re: Course back-up process failing/timing out

by An sree -


The size of the course not effected here, because we already got backup of one course it has a higher size than this one. 


Also, I have tried to create backups by unchecking all options from the backup page (Like userdata, questionsother data) but again repeat the same. 

I have a doubt that Moodle course needs any capability or any other extra settings for create backup?? 

I compared the course settings with other courses. But can't find any changes.

What do you think ?? Thank you very much for your valuable reply


In reply to An sree

Re: Course back-up process failing/timing out

by Ken Task -
Picture of Particularly helpful Moodlers

We could be guessing foreever.  Turn on debugging.   Try the backup.  Let's see if moodle reports anything.  Also check web server error logs.

If you don't do those things, I cannot attempt to help any more than I have.

'spirit of sharing', Ken

In reply to Ken Task

Re: Course back-up process failing/timing out

by An sree -

Hi Ken,

We have turn moodle debugging messages on and found warning message    " Failed to write session data (memcache). Please verify that the current setting of session.save_path is correct () in Unknown on line 0 " .  The other courses which we can create backup successfully does not show this warning  .

How can I solve this  ???






In reply to An sree

Re: Course back-up process failing/timing out

by Ken Task -
Picture of Particularly helpful Moodlers

Interesting .... don't think anyone has ever reported similar.
Googling for that error shows it happens to other apps ... not just Moodle.

https://www.google.com/search?q=Failed+to+write+session+data+%28memcache%29.+Please+verify+that+the+current+setting+of+session.save_path+is+correct+%28%29+in+Unknown+on+line+0

Possible solution ... even though the links might be about other apps:

https://ma.ttias.be/phps-memcached-sessions-failed-to-write-session-data-memcached-for-magento/

'spirit of sharing', Ken

In reply to Ken Task

Re: Course back-up process failing/timing out

by An sree -

Hi Ken,

We have fixed this issue by increasing the maximum key/slab size in Memcached. We have increased the maximum key/slab size of 10MB, instead of the default 1MB and restart the memcache service.Now the issue has been fixed. Again, thanks for your above valuable replies.


Average of ratings: Useful (1)
In reply to An sree

Re: Course back-up process failing/timing out

by Ken Task -
Picture of Particularly helpful Moodlers

Congrats!  Thanks for responding with the solution.

Maybe that should be added to Moodle docs on memcached config for Moodle! smile

'spirit of sharing', Ken

In reply to Ken Task

Re: Course back-up process failing/timing out

by An sree -

Hi ken,

I have faced one more issue that the course backup has been failed when we have selecting NONE for all user data, and the following settings at the bottom:

Users - None
Logs - None
User Files - None
Course files - Yes
Site files used in this course - Yes
Grade histories - No

But if i proceed with default settings , then it will become success. What do you think ???
In reply to An sree

Re: Course back-up process failing/timing out

by Ken Task -
Picture of Particularly helpful Moodlers

Am not even sure about what you are asking here! :\

With those settings the only thing that might bite is site files used in this course, I think.

Those settings should provide a backup that contains only content ... from what I understand.

Those take less time and are generally smaller in size and normally don't have issues.

'spirit of sharing', Ken


In reply to Ken Task

Re: Course back-up process failing/timing out

by An sree -


I have tried to create a backup of a huge course without user data.It's a very large course, almost have 6 GB. But it was failing with the following message


"An error occurred while copying the zip file to the course directory"
The backup did not complete successfully.

If we tried by less data, then we got back-up successfullyWe have found that issue is with the zipping function. It is getting aborted. This is the command that is executed in the script. While executing the same script in the command mode, the zipping process is getting aborted after 30 minutes.


we have some options or controls on extending that zipping timeout/abortion limit?

If not, then how we can get these large backups working.??



In reply to An sree

Re: Course back-up process failing/timing out

by Ken Task -
Picture of Particularly helpful Moodlers

The fact you are saying 'zip' means it's a very old Moodle ... 1.9 vintage?  Better backup/compression to .mbz files can in around 2.3 or so + there were CLI only scripts that could do course backups ... taking apache out of the loop.

Confusing ... 'while executing the same script in the command mode' ... hmmmm, customized whatever?

Which then relates to the version of PHP (guessing 5.3) thus also leads to platform ... linux or Windows?

Linux 5.3 PHP zip had a limit even with 86_64 of 4 gig and it was the processes of 'copying' the zip.  It was an operating system limitation which could be overcome, but one would have to re-compile the kernel to get the support for sizes over 4 gig.

So if I'm guessing correctly, think the only approach one can take (if stay with what you have) is to increase PHP limits to very high settings .... time for a script to run and memory used + backup courses in pieces ... like you've semi-discovered ... then, if one wants to restore that course to same system,  take the two part route ... install one backup and then the other in the same course.

Time to upgrade?   All the way round?

So I could be in-correct .... your Moodle version?  And OS?  and php version?

'spirit of sharing', Ken


In reply to Ken Task

Re: Course back-up process failing/timing out

by An sree -

We have used moodle 1.9 version , 5.2.17 of PHP version  and linux platform. I have used 64-bit system and server too.

cd '/campus-data/campus-name/temp/backup/1533299572' ;'/usr/bin/zip' -r '/campus-data/campus-name/temp/backup/1533299572/algebra_1_-20180803-0632.zip' 'moodle.xml' 'group_files' 'course_files' 'moddata' 'site_files'

Please look at the above command. This is the command that is executed in the script and the zipping process is getting aborted.If we executed the same command in the command line, then we got a successful backup zip file after 30 minutes.

We have set the following code on backup/backup.php file 

@ini_set('max_execution_time', 0);

raise_memory_limit("192M");

set_time_limit(0);

So we hope PHP limits not affect our normal backup process.

Is it possible to create the backup of large sized course in moodle 1.9 version  ?? 

In reply to An sree

Re: Course back-up process failing/timing out

by Ken Task -
Picture of Particularly helpful Moodlers

What you are doing from command line mimics what the web based routine does for creating the backup file, but NOT the last step ... which is to 'copy' the backup zip file in temp to a course ID/backups/ directory.

And that's where it was discovered the issue ... had the same issues back then (it's been years since I had to contend with a 1.9.20+ version) and discovered the web based failed ... every time ... I could see a valid backup zip file in /temp/...  using the cp command from the command line failed.  Huh?!!!  but the 'mv' command didn't.

The issue really a kernel thang ... how large a file 'cp' could work with - CentOS 5 at the time, and found info in sys admin blogs/etc. that the limit for cp was 4 gig.   Could get the OS to do more, but it would mean re-compling the kernel to get the ability.   Uhhh ...  no thanks!!!   The 'mv' command isn't restricted like 'cp' is.  So when I ran into one of those courses (50+Gig if I re-call correctly) I manually moved the .zip to the appropriate location where it could seen in the Moodle interface.   Natch ... restoring that course was an issue then.

Guess you could hack code and see if you could get a 'mv' routine to run as opposed to a 'cp'.

OR, one could bite the other bullet and upgrade Moodle - which started using gzip for better compression etc..

Good luck!

'spirit of sharing', Ken


In reply to Ken Task

Re: Course back-up process failing/timing out

by An sree -

Hi Ken,


First of all , sorry for the late reply.

we have find out the solution for this problem.For compressing files, Moodle uses either the PclZip library or the external zip program. Our campus has used an external zip program which is faster than PclZip method but it could get aborted may be due to kernel limitations. We have created the backup successfully by using PclZip method . 

For this, we have changed the settings called "Path to zip" in Site administration -> Server-> System Paths which indicates the location of  zip program. It is empty in default condition but this campus is already set this path (/usr/bin/zip). We have made this empty and then, Moodle will use PclZip library instead the external zip program. It will take more time to zip the file. 

In reply to An sree

Re: Course back-up process failing/timing out

by Ken Task -
Picture of Particularly helpful Moodlers

Glad you have resolved the problem.   Encourage you to think about upgrading as 1.9.x is very, very old.  You've found a solution for now, but future?

'spirit of sharing', Ken


In reply to Ken Task

Re: Course back-up process failing/timing out

by An sree -

We could face any problem while upgrading 1.9 to latest moodle ? what you think .

In reply to An sree

Re: Course back-up process failing/timing out

by Ken Task -
Picture of Particularly helpful Moodlers

Well, yes ... 1.9 -> now 3.5 ... there are quite a few versions and changes ... 15 major versions and more 'point releases' (those last .#'s) between where you are and the latest/greatest.

Might want to strategize ... do a 'moodle march' or NOT do a 'moodle march'.  

Not trival.

Lots to look at cause it involves upgrading PHP/MySQL .... and am almost willing to bet, the version of your server as well ... like Ubuntu 14.04 to highest ... or CentOS 5 to CentOS 7 ...

For Moode check out:

https://docs.moodle.org/dev/Releases

Like I said, lots to read and study.

'spirit of sharing', Ken