Can't delete user backup files

Can't delete user backup files

by Mike Hanratty -
Number of replies: 8

In the user backup area there are about a dozen files (close to 1G). When I click on Manage Backup Files I get an error that says "Cannot read file ....."  When I click individually on the "Download" links all but one can be down loaded.  This file gives an error "firefox can't find the file ....." I assume this file is corrupt but it is causing major problems as the backup size is getting out of hand and I can't delete any of them

I am using v2.4.1 and am logged in as a full administrator.  I just ran cron.php and that made no difference.

Average of ratings: -
In reply to Mike Hanratty

Re: Can't delete user backup files

by Ken Task -
Picture of Particularly helpful Moodlers

The error says a possible problem with 2 hints ... the latter is a permissions issue.  What are ownerships/permissions on the moodledata/filedir directory and directories contained therein?

The page which shows the files is rendered from the database ... it is not generated by looking at the actual folders and files (new file system).

Files in the new file system are no longer in human readable form, but rather, a hash named file.  The is no file browser ... not even for System Admin (something Moodle needs to provide).

Only other way ... mysql query - using phpmyadmin or other MySQL tool:

select * from `mdl_files` where `filename` like "%.mbz%"

WIll show all references in the DB for a Moodle 2 backup file - they have a filename extension of .mbz (really a zip).

In the output of the query above, one will see a contenthash column with references like this (example):

b9d74f3c79e504e5685b1558a6384430a6361286

How to use the above info ... with any tool you have to browse files, go to your moodledata folder filedir directory.

Contained therein one will find (as per example above) a b9 directory.  Contained in the b9 directory a d7 directory.  In the d7 directory there will be a file with the name:

b9d74f3c79e504e5685b1558a6384430a6361286

Command line view in this example:

[root@sos filedir]# cd b9
[root@sos b9]# ls
1d  2c  41  d7
[root@sos b9]# cd d7

/var/www/moodle24data/filedir/b9/d7

[root@sos d7]# ls -l
total 1028
-rw-rw-rw- 1 apache apache 881134 Dec 22 19:09 b9d7235573ec258426a2f407733ace3a11d600e7
-rw-rw-rw- 1 apache apache 155763 Jan 30 07:28 b9d74f3c79e504e5685b1558a6384430a6361286

In the example above, one can see the files are owned by apache user and group.  So in this case the pages generated that provide links to them will be found.

One of them, is a moodle 2 backup (.mbz ... which is really a .zip):

[root@sos d7]# file -b b9d74f3c79e504e5685b1558a6384430a6361286
Zip archive data, at least v2.0 to extract

Sorry for the tutorial on how the new file system works (in this case doesn't work).  It might have been enough to simply say ... check permissions/ownerships on moodledata/filedir ... but if there are other file system issues which might arrise later, at least you kinda know where to look and how now.

'spirit of sharing', Ken

Average of ratings: -
In reply to Ken Task

Re: Can't delete user backup files

by Mike Hanratty -

Hi Ken, this is the first time I've seen this problem. This is an old, old moodle that has been running for years, updated regularly.  It is running on SuSE server where moodledata is in /srv/www.  Everythig else is working perfectly

The odd thing is that is effecting only one file in the backups.

I followed your suggestions and found the contenthash for the offending file:

2e96b45149355d189ec0251cdd46564dfff19d61

I went to ../2e/96 but it was empty.  Of all the directories in ./2e, 96 had permissions of 777.

Weird!

Thanks, Mike

Average of ratings: -
In reply to Mike Hanratty

Re: Can't delete user backup files

by Ken Task -
Picture of Particularly helpful Moodlers

So it's truely missing!!!!  About the only way I can think of where that *might* happen is if one had keep 5 backups in settings somewhere and then reduced the number to keep to a lesser number.  At some point in time the file was moved to the trashdir and 4 days later or so the cron job emptied the trash.

That should have also cleaned up the record in the DB as well but didn't.  Got any others like it?

Here's a little script I created to help me 'explore' the filedir of Linux based Moodle.  Might help ya discover other files.

ls /var/www/;
echo 'Usage: explore [moodledata] directory located in /var/www/[moodledata]/filedir';
echo 'Working in /var/www/'$1'/filedir';
echo 'CNTL C to CANEL.  Enter to execute.';
read $keypress;
cd /var/www/$1/filedir;
for f in ./*/*/*; do echo "$f"; file -b "$f"; echo "-------"; done

output looks like:

-------
./d1/c9/d1c9e2278049d68e68ce07b1454f410af3c5fc02
Zip archive data, at least v2.0 to extract
-------
./d1/ea/d1eadfba68c5df4972c652f945fe18db27c73353
Microsoft Office Document
-------
./d3/ea/d3eac2e2dbe5eee784d34c48d1d71f33710815d5
Zip archive data, at least v2.0 to extract
-------

Script could be modified to limit out put to finding 'Zip archive data' only ... which would be either .zip files or .mbz's.

Question is ... how did the hash file disappear?

I take it that all other backups are working.

'spirit of sharing', Ken

Average of ratings: -
In reply to Ken Task

Re: Can't delete user backup files

by Mike Hanratty -

All other backups are working fine!

Average of ratings: -
In reply to Mike Hanratty

Re: Can't delete user backup files

by Mike Hanratty -

I have just deleted the offending entry in the files table in the database and the file listing has disappeared from Restore listing but there is still the same error when I click Manage files.  I have tun cron.

Average of ratings: -
In reply to Mike Hanratty

Re: Can't delete user backup files

by Mike Hanratty -

I have investigated a little further.  I have tried to download the remaining files.  The first 5 actually start to download and because they are relativly large I cancelled the downloads.  All the remainig files even though they are listed as having sizes of many MB, all complete their downloads in a few seconds and all have the identical downloaded size of 97.3kB.  Very weird.

So, when I get time I am going to try and find each of these newly offending files in the database as per Ken's instructions above then delete each file if it exists then delete the database entry.

I'll post here again when I have given it a try. Could be as late as Friday.

Even if it solves the problem, the solution will open more questions than it answers!!!!!!!

Average of ratings: -
In reply to Mike Hanratty

Re: Can't delete user backup files

by Ken Task -
Picture of Particularly helpful Moodlers

Backups of that small size is really not that unusual IF there were 'test courses' ... no students assigned and no activities ... just a shell course.

In 1.9.x on CentOS (other Linuxes as well) there was an issue of creating a large backup (over 4.0Gig) and when that operation was completed 'copying' the backup (.zip) to course ID/backupdata/.  Because that was the last operation of the entire process, Moodle would record the backup failed.  But it didn't really.  When inspecting /moodledata/temp/backup/ one could find a completed/intact backup.  Operating system just couldn't copy.  Linux was capable, but one had to compile supports for such large files into the kernel.   Uhhh, something I'd care not to do.  One could, however, manually move the backup to it's proper location.

Check your moodledata/temp/backup/ folder for anything remaining.If the course backups reached that threshold there might be some .mbz files there.

Also, there might be .log files larger than 0 bytes.  0 byte files indicated successful backups.  Any .log file larger than 0 bytes might also give a clue concerning backups.

'spirit of sharing', Ken

Average of ratings: -
In reply to Ken Task

Re: Can't delete user backup files

by Mike Hanratty -

The really odd thing is that all these "failed" backups were for real classes that I was copying for other courses and actually restored earlier on.

Average of ratings: -