Delete old Backups after setting up individual "backup | backup_auto_destination"

Delete old Backups after setting up individual "backup | backup_auto_destination"

by Nicolas Romero -
Number of replies: 6

Dear colleagues,

until last week we where saving the automated backups in the defalut directory (moodledata/filedir).

We decided to set up a new individual directory (due to storage optimizing). So far so good, the automated backup process did work fine and saved all the recently updated courses in the new directory. (Btw: "backup | backup_auto_storage" is only set to "specified directory for automated backups").

But it seems, that the old backups in the filedir directorys are not deleted by the automated backup process. "backup | backup_auto_max_kept"=1, so i expected, that backuping a course in the new directory would delete the old one in filedir.

But it does not. So i still have all the old backups. Now the question: any idea how to delete the old backups? (besides manually?...over 1500 courses)

I did not find any helpful thread. If you know one plz let me know.

Kind regards,

Nick



Average of ratings: -
In reply to Nicolas Romero

Re: Delete old Backups after setting up individual "backup | backup_auto_destination"

by Ken Task -
Picture of Particularly helpful Moodlers

What version of Moodle?

Are you sure backups still reside in moodledata/filedir/

select contenthash,filename,component,filearea from `mdl_files` where (`component` like "backup" and `filearea` like "automated")

+------------------------------------------+----------+-----------+-----------+
| contenthash                              | filename | component | filearea  |
+------------------------------------------+----------+-----------+-----------+
| da39a3ee5e6b4b0d3255bfef95601890afd80709 | .        | backup    | automated |
| da39a3ee5e6b4b0d3255bfef95601890afd80709 | .        | backup    | automated |
| da39a3ee5e6b4b0d3255bfef95601890afd80709 | .        | backup    | automated |
| da39a3ee5e6b4b0d3255bfef95601890afd80709 | .        | backup    | automated |


Note that all the contenthashes are exactly the same .... below you'll see why.

Then from command line ...
cd /path/to/moodledata/filedir/

Find for file by contenthash:
[root@sos filedir]# find ./ -name da39a3ee5e6b4b0d3255bfef95601890afd80709
./da/39/da39a3ee5e6b4b0d3255bfef95601890afd80709

Check for file info:
[root@sos filedir]# file -b ./da/39/da39a3ee5e6b4b0d3255bfef95601890afd80709
empty

note is says empty.

checking that:
[root@sos filedir]# ls -l ./da/39/da39a3ee5e6b4b0d3255bfef95601890afd80709
-rw-rw-rw-. 1 apache apache 0 Nov 17  2017 ./da/39/da39a3ee5e6b4b0d3255bfef95601890afd80709

shows 0 bytes.

It's a place holder, so to speak.

Those are automated backups that are working as expected.

select contenthash,filename,component,filearea from `mdl_files` where `component` like "backup";

+------------------------------------------+----------+-----------+-----------+
| contenthash                              | filename | component | filearea  |
+------------------------------------------+----------+-----------+-----------+
| da39a3ee5e6b4b0d3255bfef95601890afd80709 | .        | backup    | automated |
| da39a3ee5e6b4b0d3255bfef95601890afd80709 | .        | backup    | automated |
| da39a3ee5e6b4b0d3255bfef95601890afd80709 | .        | backup    | automated |
| da39a3ee5e6b4b0d3255bfef95601890afd80709 | .        | backup    | automated |
| da39a3ee5e6b4b0d3255bfef95601890afd80709 | .        | backup    | course    |
| da39a3ee5e6b4b0d3255bfef95601890afd80709 | .        | backup    | course    |
| da39a3ee5e6b4b0d3255bfef95601890afd80709 | .        | backup    | course    |
| da39a3ee5e6b4b0d3255bfef95601890afd80709 | .        | backup    | course    |
+------------------------------------------+----------+-----------+-----------+

If backups were made by other means, filenames would not be a '.' but something like

backup-moodle2-course-3-mshare-20180701-1938.mbz

IF you find other backup files in moodledata/filedir/ could they be backups made by teacher?

'spirit of sharing', Ken

Average of ratings: -
In reply to Ken Task

Re: Delete old Backups after setting up individual "backup | backup_auto_destination"

by Nicolas Romero -

Hi Ken,

moodle version 3.5.

Thx for the reply, i´ll give it a try and feedback what i found out.

Kind regards,

Nick

Average of ratings: -
In reply to Ken Task

Re: Delete old Backups after setting up individual "backup | backup_auto_destination"

by Nicolas Romero -

Using select contenthash,filename,component,filearea from `mdl_files` where (`component` like "backup" and `filearea` like "automated")

I get lines like this:

da39a3ee5e6b4b0d3255bfef95601890afd80709| .| backup| automated
(this files are empty)

And lines like this:

1194525812deb86cb0ed44724b2c395e6411a69f| sicherung-moodle2-course-1-moodle-20180830-0055.mb...| backup| automated

This second type of files are not empty. They have individual contenthash values. And the filenames correspond zu the automated backups listed in the restore site of the corresponding courses. So no manual backups.


Let me summary some findings:

  • In filedir i can find backups of all courses using the contenthash from the table (besides the place holders)
  • the files are not empty
  • In the new backup_auto-destination i can find all courses that have been backuped since i added the individual new folder.
  • This means, that for some courses i have two backups, one in the new folder, and one in the old default location. The filenames are the same in the table, the new folder and the course restore backend.
  • (Deleting the automatied backup in the course backend does not delete neither the backup in filedir nor the backup in the new folder) (<- was just testing)

Average of ratings: -
In reply to Nicolas Romero

Re: Delete old Backups after setting up individual "backup | backup_auto_destination"

by Ken Task -
Picture of Particularly helpful Moodlers

So it appears that changing the config of autobackups to a designated directory only didn't clean up auto backups that existed in /moodledata/filedir/.   Right?

This one:

"(Deleting the automatied backup in the course backend does not delete neither the backup in filedir nor the backup in the new folder) (<- was just testing)"

When one deletes files via Moodle UX, they get moved to trashdir and are finally erased when cron job sees (4 days later?) files in trashdir have not been relinked.

So what does a cron run/task (run now) for trashdir look like?

Not sure moodle can find and clean up automated backups after a change of config like that ... that's why in the other posting you found, moosh was suggested.  If I re-call correctly, moosh does two things ... cleans up the rows in mdl_files *and* removes the files from /moodledata/filedir those rows referenced as well.  

One cannot avoid interacting with DB tables in this case, me thinks ... one doesn't want 'orphaned' references in DB or 'orphaned' files physically present in moodledata/filedir/

BTW, a backup could exist in teachers private files area.   Admin's cannot see those unless one logs in as the teacher, *pretends* to restore a course.  Can then manage backups seen in private files (ie, delete) and then cancel the restore.   That could be too much for admin to do ... thus moosh.

Me thinks you could modify the query to seek what backup files reside in 'private' areas - I'll let you tinker with that! smile

'spirit of sharing', Ken


Average of ratings: Useful (1)
In reply to Ken Task

Re: Delete old Backups after setting up individual "backup | backup_auto_destination"

by Nicolas Romero -

All right, thx for the explanations. cool

I´ll try some other querys and will check out moosh. In fact, from a first look moosh seems to have all sorts of nice commands.

Will post in here my findings.

Kind regards,

Nick

Average of ratings: -