Bulk Archiving Courses

Re: Bulk Archiving Courses

by Ken Task -
Number of replies: 5
Picture of Particularly helpful Moodlers

You might check into installing moosh - CLI only.   It can be scripted.

See: https://moosh-online.com/commands/

Several commands for which there is no equiv in the Moodle Admin UI.

'spirit of sharing', Ken

In reply to Ken Task

Re: Bulk Archiving Courses

by Sean Scott -

Hi,


I have been using the information Ken Provided for bulk archiving. Now I'm trying to do the same on an older version of Moodle 2.6.3. However there is not backup.php script in this version only an automated_backup.php script.

Can anyone provide some additional details as to how to perform bulk archiving in an older version of Moodle.

Can I copy the backup.php script from a newer version and use it on Moodle 2.6.3 without issue?


Any additional help is appreciated.

Thank you,

In reply to Sean Scott

Re: Bulk Archiving Courses

by Ken Task -
Picture of Particularly helpful Moodlers

Yep ... 2.7 was when the cli backup.php script first appeared - none for 2.6.x.  Don't have a functioning 2.6 up anywhere to do a quick test soooooooo ....

And, yes, you could try that ... copy a 2.7.x backup.php script into admin/cli/ of the 2.6.

And try it ... think I'd try course id 1 (that's the front page - which you can't  restore anyway) cause it's generally small as far as a backup is concerned *and* use the option to save into destination directory .. *NOT* filedir.   Make sure your  2.6 is set to use what it called 'experimental compression' - gzip instead of zip.   That will make your manually created backups forward compatible - or less prone to issues ... me thinks.

If that doesn't work, get moosh ... but here again, one might need to install a version of moosh that isn't too high ... can't tell you which one, but here's the link:

https://moodle.org/plugins/pluginversions.php?id=522

I can re-call tinkering with that ... I did have multiple versions of moosh on same server ... fairly easy to do because the command you create to use is an alias ... had a 'moosh' which pointed to the highest version ... and a 'moosh29' which was aliased to the 29 version of moosh.

The actual moosh directories changed slightly ... moosh and moosh29.

'spirit of sharing', Ken

In reply to Ken Task

Re: Bulk Archiving Courses

by Sean Scott -

Hi Ken,

Can you tell me how to modify the script to use --courseshortname instead of --courseid


I'm assuming the $i needs to be modified since course shortname is alpha numeric. Just not sure how to do it. 

Thank you for your help.

This is the script that works with the --courseid

#!/bin/bash

#

cd /ctdlc/moodle/admin/cli/;

echo pwd

for i in $(cat /ctdlc/moodle/admin/cli/2017Summer1.txt)

do

    echo "Course ID in que:" $i;

    php backup.php --courseid=$i --destination=/mnt/nas/archives/

done

ls -l /mnt/nas/;

echo 'Done!';

I would like to use --courseshortname

#!/bin/bash

#

cd /ctdlc/moodle/admin/cli/;

echo pwd

for i in $(cat /ctdlc/moodle/admin/cli/shortnameSU17.txt)

do

    echo "Course Shortname in que:" $i;

    php backup.php --courseshortname=$i --destination=/mnt/nas/shortnameSU17/

done

ls -l /mnt/nas/;

echo 'Done!';


In reply to Sean Scott

Re: Bulk Archiving Courses

by Sean Scott -

Hi,

I'm also noticing that although I'm using the --destination option in my backup script.. The partition I have Moodle installed on continues to grow rather large after each set of backups I perform.

Do you know if there are copies of these courses being copied elsewhere other than the destination I'm sending them to?

The last time I ran the script I ran out of space.

I do see a bunch of log files in the moodle/backup/temp directory but that doesn't appear to be taking up the space that decreasing during each large backup job.

Thank you,

In reply to Sean Scott

Re: Bulk Archiving Courses

by Ken Task -
Picture of Particularly helpful Moodlers

Hmmmm .... let's see ... we have a 2.6.x and using a script that first came with 2.7.x.  While one could type:

php backup.php

and see the options ... of which --courseshortname=STRING is an option displayed, it's still a script for a higher version of Moodle than what you are using on 2.6.   Short answer (no pun intended): nope.   I cannot tell you how to get that working with 2.6.x.    But feel free to hack away! ;)

Did you try it?   Bet 'STRING' needs to be enclosed in quotes as many folks just copy the title of the course into short name ... and don't think I'd like that cause that would mean they also have 'spaces' and other characters in name of the backup file that might actually be mis-interpreted by bash (like \) or I have to use quotes around those filenames when using those files with other file operations.  In other words, I'd advise not using shortnames.

In the preferences for backups of a 2.6 (site admin), aren't there options for what to use in the backup filename?  Not only the course id but also the short name in the backup file name?    Check that.

We'll do two in one response ... to the best of my knowledge only moodledata/backup/temp/ is used when making backups.   And if the destination option to the cli script is working and you see that /mnt/whereever filing up with .mbz files and no folders/directories remain in moodledata/backup/temp/ it's doing what it should.

Do you have automated backups turn on or off?

Are you running cron job often enough? (think the recommendation today is once every minute).   The moodledata/trashdir/ might fill up for a while until cron runs to clean it up.  Or, you need to break apart your courseidlist.txt file ... fewer course ID's listed ... fewer backups.   That would give you time to go into moodledata/trashdir/ and manually remove all files/directories.

Cannot do Vulcan Mind Meld with your 2.6 from here ... but can Google ... and lookie what Google found for:

"linux finding largest files on a system". ;)

https://unix.stackexchange.com/questions/140367/finding-all-large-files-in-the-root-filesystem

'spirit of sharing;, Ken