automated backup erorr / cron setup

automated backup erorr / cron setup

by Michael Rodgers -
Number of replies: 16

Hi all, I'm running Moodle 2.6.6 and am having some teething problems running a full site backup.  From reading the documentation http://docs.moodle.org/22/en/Automated_course_backup , http://docs.moodle.org/22/en/Cron_with_Windows_OS and http://docs.moodle.org/22/en/Cron my understanding is that Cron is something that runs in the background and keeps Moodle ticking over and maintaining the system.

I am wanting to schedule a full course backup and so through site admin > courses > backups > automated backup setup

I have set this to active and have got every day ticked with a time of 8pm in the evening, I specified the 'course backup file area and the specified directory' as the save location as well as creating a folder called moodlebackup on my C:\ and provding its filepath in the save to box underneath, so C:\moodlebackup.  There is a tick next to this box which according to instructions indicates a correct file path specified. Everything is ticked under 'automated backup settings' and use course name in file backup is also ticked.

After saving this and letting the backup run and then going to my backup log, every course says error next to it.

I discovered that the initial automated software exe cron although setup had not been running correctly and so with some extra work as per http://docs.moodle.org/22/en/Cron_with_Windows_OS the command line version was implemented.  This is linked to a bat file that also logs the cron output, similar to this line

c:\php\php.exe -f c:\moodle\admin\cron.php > c:\moodle\admin\cron.log

Now after this runs it produces the message 'skipped' for each course.  I'm aware that skipped means if a course hasn't been changed for 31 days it will be skipped but I know that some of them, maybe 4/5 are altered on a weekly basis so can't work out why they are being skipped. 
The log file is reporting that automated backup has been scheduled and lists every course as part of this backup but then outputs the message automated backup successful. I have searched for .mdz(?) files which I beleive are the Moodle backups but can't find anything and also if I go in any course and then click the restore icon for it to look at the backups, again nothing is listed.

I'm unsure over why all courses have a skipped status when I know that some have been edited recently.

Also has the automated backup been incorrectly implemented and how does this differ from the scheduled cron backup. Will they not conflict?

Thanks in advance for any replies on this matter,
Michael
Average of ratings: -
In reply to Michael Rodgers

Re: automated backup erorr / cron setup

by Ken Task -
Picture of Particularly helpful Moodlers

How many backups are to be retained?  That setting supposedly applies to backups regardless if automated or 'normal'.

See if it's really working:  move all the backups in C:\moodlebackup out into another folder.

In the admin/cli/ folder of that version there is a automated_backups.php script.  For testing, might try to use that rather than cron.

Then run the cron.

'spirit of sharing', Ken

Average of ratings: -
In reply to Ken Task

Re: automated backup erorr / cron setup

by Michael Rodgers -

Morning John the issue is that no backups are being created at all hence the e-mail. Once the cron has run it says the auto backup has been correctly run yet in settings > reports > backups it says skipped.

Average of ratings: -
In reply to Ken Task

Re: automated backup erorr / cron setup

by Michael Rodgers -

Hi again Ken, I have been doingsome further experimenting.  It seems as though Moodle is skipping all courses regardless of old or new.  I have just created two new courses myself, test course 1 and test course 2, assigned two students to each one and added some documents into each course.  I amended the automated backup schedule time to allow me to check what has been backed up.  Even after letting this run and then going  into the log area the two courses (along with the few existing ones that have been working on in the last week) still shows as 'skipped' inthe backup log.

 

I have checked the cron log file on the server and it shows automated task has run and the two courses are showing in the log but no backup has been created.  May you have any further suggestions please?

Thank you,

Michael

 

Average of ratings: -
In reply to Michael Rodgers

Re: automated backup erorr / cron setup

by Ken Task -
Picture of Particularly helpful Moodlers

Recheck permissions on moodledata/temp folder.

Check apache error logs.

Check the moodledata folder temp/backup.

In that folder one might find 0 byte files named with hash.  Something like: 18b794c41782d9d6e0ce13fe69f78762.log

Those are indications of successful backups.   There may not be any of those files.

If there are such files with any byte size, those are failed backups.  Since they are text, open with text editor.  They might say something about Moodle not being able to find a handler for X.  That will give you a clue.

Also, there might be folder with a hash name like:

18b794c41782d9d6e0ce13fe69f78762

These are the temporary folders created by Moodle for building .mbz backups (.mbz backups are really .zips).  Explore those.  To see humanly understandable file names one will have to cross reference the files.xml file with the contents of the files directory found in the temp archive.

If one finds what appears to be completely constructed folders or a .mbz file still there, it might be an issue related to OS/php not being able to move/copy a large backup to the other designated area.

Those temporary files/folders in backup can be removed manually (safely).

If one finds nothing that stands out, consider turning on debugging and run a course backup manually.

In http://docs.moodle.org/22/en/Automated_course_backup

there is a note:

Note: Automated course backups automatically skip courses which are unavailable to students and have not been changed in the last month.

Have you tried the automated_backup.php script in admin/cli/ yet?  That doesn't involve any cron and restricts action to just that process.

'spirit of sharing'. Ken

 

Average of ratings: -
In reply to Ken Task

Re: automated backup erorr / cron setup

by Michael Rodgers -

Good afternoon Ken, thank you for this update.

I have not started to look through your solutions as yet but when trying to create a manual backup for a course instead, the system produced the following error 'error/baseelementchildnameconflict'. I am wondering if this problem is related to the automated procedure which would explain why both ways are not working?

 

Thanks,

Michael

Average of ratings: -
In reply to Michael Rodgers

Re: automated backup erorr / cron setup

by Ken Task -
Picture of Particularly helpful Moodlers

Yep!  That would cause errors in automated backup also!

http://tracker.moodle.org/browse/CONTRIB-3534

What is the default course format?  Topics/Weekly?  Are you using an add-on course format?

Any chance of upgrading the site to at least the highest 2.3.x? Your current version 2.2.6 is next to 'slide' into 'Supported for security fixes only'.

BTW, I know this doesn't help you, but I do have a 2.2.6 site using automated backups and it functions just fine.  Site is Linux OS, however.

'spirit of sharing', Ken

Average of ratings: -
In reply to Ken Task

Re: automated backup erorr / cron setup

by Michael Rodgers -

Hi Ken, thank you for the clarification.  Indeed your reply has helped to explain the past two days performing tests, more tests and checking over log files.

I have checked the link you send me and admittedly from the title it appears to relate to restoring courses in a particular format but guess it is related.

The current selected course format is topics although to the right of this it says the default format is weekly. There are no add-on course formats that I know about, in fact I don't know what one is really.

Upgrading is something that I have already been looking into as I'm aware the version is fast becoming outdated.  I have previously tried this but made the mistake of using the copy / paste method, trying to overwrite the old moodle directory with the new one and it complained about having two significant versions of Moodle running at the same time. After some research I discovered that best practice is actually to start from fresh, install the latest set of moodle files and then add any additional blocks to this moodle install so that everything remains compatible.  The issue with this is that because we are in a school environment in the UK we are running a third pary block that allows for Moodle to connect to our school course management system (Eportal / CMIS if you have ever heard of this) which displays all the course available to students within Moodle.  I am okay installing the new version (was 2.3 but it now 2.4) but there is a configuration issue on our side involving this third party block which is preventing me so far.

 

Out of curiosity our institute has also got another instance of Moodle setup which has just been upgraded from 2.3 to 2.4 but I tried the same automated backup test yesterday on this and it produced the same result, newly created course skipping?  I didn't attempt a manual backup yesterday but have just tried this and it was successful.  The setting for courses in this install is weekly, the same as the default. 

I mention this as your previously reply indicates two possible reasons for failure, one being the moodle version and the other the format of the course display? If on version four the manual backup works and courses are wet as default to weekly I would have expected automated backups to also work?

 

Thank you for being very informative, it really is appreciated.

Michael

Average of ratings: -
In reply to Ken Task

Re: automated backup erorr / cron setup

by Michael Rodgers -

Hi Ken, sorry to ask again. I was just wanting to know what the correlation is between my environment, the problem being experienced and your diagnosis post above?

Thanks,

Michael

Average of ratings: -
In reply to Michael Rodgers

Re: automated backup erorr / cron setup

by Ken Task -
Picture of Particularly helpful Moodlers

First, am not a programmer ... just an 'informed' user/server admin with 'seat of the pants experience' who attempts to resolve issues involving Moodle.  Sometimes, am nothing more than a sounding board or a provider of thought.

Not sure what is meant by:

'correlation is between my environment, the problem being experienced and your diagnosis post above?'

Diagnosis of a site un-seen, you mean!  That's a pretty tall order and I've lost my magic wand ... actually never had one!  ;)

Have no first hand knowledge of the CMIS add-on and how that might affects backups - manually or automated.  Is the CMIS system used for authentications?  That means users are coming from that system and are not really local (to Moodle accounts) ... much like LDAP.  In the setup of backups, are users included?  Does that mean the backup routine has to communicate with the CMIS system to acquire users to place in backups?

While you've described (very well) your situation, only if someone else has pretty much your exact setup could a more specific answer be provided, thus would suggest other users of the CMIS plugin you've mention get together with the maker of the CMIS plugin to discuss issues/strategies for moving forward (ie, upgrading).  If there are many in UK, by all means find out who else uses and inquire if they are having the same issues.

Am almost certain that things like checking logs, turning on debugging etc. will provide clues, but it would be up to someone familiar with the plugin/your system to be able to interpret errors and strategize an approach to fix.

Sorry can't be of more help than that!

'spirit of sharing', Ken

Average of ratings: -
In reply to Ken Task

Re: automated backup erorr / cron setup

by Michael Rodgers -

Hi Ken, the information about CMIS was just to provide background details about why I've not upgraded Moodle to version 2.3 just yet. The add-on itself isn't used for authentication (it is used to retrieve course information) but it does run in conjunction with LDAP, so as you correctly point out, the user accounts are not stored locally.

My query "correlation is between my environment, the problem being experienced and your diagnosis post above?'"  relates to the association between: -
the link that you provided me which makes reference to the error mesage that I initially posted,
the questions that you asked i.e. course default setting whether weekly or topics,
the inability for Moodle to perform automatic backups and how all of this fits together i.e. you identified that the error message from my manual backup attempt was also the reason why auto backups didn't work either. How does the link join all these parts together? Is there a bug when wanting to backup courses with specific fornat setting on my version etc?

Thank you,

Michael

Average of ratings: -
In reply to Michael Rodgers

Re: automated backup erorr / cron setup

by Ken Task -
Picture of Particularly helpful Moodlers

Probably should have referred to the following URL to begin with:

http://docs.moodle.org/24/en/Backup_FAQ

While it says 24 in the URL, some of those items I know are for 1.9's as moodle.xml file is no longer used in 2.x backups (as an example).

'spirit of sharing', Ken

Average of ratings: -
In reply to Ken Task

Re: automated backup erorr / cron setup

by Michael Rodgers -

Thank you again for your reply and the link, however I'm still unsure how this relates to the linkin your post prior to this.  I know the page refers to some error codes and problems experienced but don't seem to be related to the one I had.

http://tracker.moodle.org/browse/CONTRIB-3534

where you also mention default course settings and the current moodle version.  Could you clarify this for me please if possible?

Thank you,

Michael

Average of ratings: -
In reply to Michael Rodgers

Re: automated backup erorr / cron setup

by Ken Task -
Picture of Particularly helpful Moodlers

Were any courses backed up manually before the attempt to setup with cron + automatted backups?  in a course itself ... backup link in the course admin menu.  Think your last post said you tried that and it worked.  Hmmmmm ....

This problem started with one setting up cron on a system that doesn't know about 'cron' with:

c:\php\php.exe -f c:\moodle\admin\cron.php > c:\moodle\admin\cron.log

Once setup, the courses weren't being backed up.  So have you tried the command above from the command line without the redirect output going to a cron.log file?  The batch file setup for that runs as what user? 

If one were to just use the browser ... http://yoursite/admin/cron.php what happens?  Run that way, cron.php execution is by the web server user, is it not?  (same as being logged on as a teacher and teacher clicking the backup link in course menu).

On this page:

http://docs.moodle.org/23/en/Cron_with_Windows_OS

it does show some alternatives.  Can only suggest one tries something new if what's there isn't working!

'spirit of sharing', Ken

Average of ratings: -
In reply to Ken Task

Re: automated backup erorr / cron setup

by Michael Rodgers -

I never attempted any manual back ups, instead I went straight into scheduling automated backups. Cron was the very last thing to be implemented as per the setting up moodle guidelines from this site as itdoes say that Cron is very important and must not be missed.  The instructions were followed as per in the Cron Windows link above and other related links from this page.

One of my previous posts did say that a manual backup had been tried (and was successful) in another moodle testing environment (moodle2) that we have running. This was using 2.3 but has since been upgraded to 2.4 when this became stable. This is using the same system architecture as the current moodle version (moodle1) which has the backup issue.

Since our conversations I have logged on to moodle2 and noticed that the backup log for the scheduled automated backups is reporting an OK status (the automated backup is implemented in exactly the same way and uses the same cron method as that of moodle1) - it must have just needed time to let it run rather than forcing the issue.

I did also try running the cron via web interface but this didn't work either.

I'm a little confused, going back to one of your posts a few posts back where you provided a link to a known issue in moodle as well as asking about how the course format was setup.  I was of the impression that you were aware of a link between the error message 'error/baseelementchildnameconflict', the moodle version being used and how my courses are setup, hence asking the questions you did. Is there such a connection as from reading the bug explanation, it seemed to refer to restoring rather than backing up?

Thank you again.

Michael

Average of ratings: -
In reply to Michael Rodgers

Re: automated backup erorr / cron setup

by Ken Task -
Picture of Particularly helpful Moodlers

'I was of the impression that you were aware of a link between the error message 'error/baseelementchildnameconflict', the moodle version being used and how my courses are setup, hence asking the questions you did. Is there such a connection as from reading the bug explanation, it seemed to refer to restoring rather than backing up?'

Sorry if you got the wrong impression ...  what I did was attempt to help.  Did a google search for 'error/baseelementchildnameconflict moodle' and got the hit for the tracker posting.  Read the tracker posting and while it sounded similar it may not have been.  It did refer to a course format issue.  From previous experience, know if a backup putzes, restore may not work. 

I do NOT know ... first hand ... if the tracker item is related to your issue. So let's forget that!  Have never seen nor experienced the same error (but then again, don't run Windows). 

Since there is no Windows specific forum now, Windows users will post into what forum seems appropriate to them.

Ok, Windows users ... am bumping this to you all!  If you run Windows, help Michael!

'spirit of sharing', Ken

Average of ratings: Useful (1)
In reply to Ken Task

Re: automated backup erorr / cron setup

by Michael Rodgers -

In fact you have been a great help by highlighting related issues and also pointing our areas for further research.

I'm just unsure if the problem is a one off, if something has become corrupt in the current Moodle code, if another install of the same environment on the same Windows platform would resolve the issue or if its just a bug with what is now an old version of Moodle that could be caused by a various reasons that will be resolved in the newer versions.

Admittedly I have only seen the error mesage in its initial form, I haven't turned debugging on or any other troubleshooting techniques and procedures to see if it could be quickly resolved by some means, I just thought I would post the message to see if it was already known.

As a bit of something extra I came across
https://moodle.org/mod/forum/discuss.php?d=200498 and https://moodle.org/mod/forum/discuss.php?d=200601.
This has a link to the the info you previously provided me with.  They specific the problem being with grid format and refer to it as being a plugin. As far as I'm aware, no extra course plugins have been installed on my installation if this does turn out to be the issue. I didn't even know that extra course formats could be added on top of the standard install?

Thank you for your time and patience,

Michael

Average of ratings: -