the story of the misteriously filling up hard disk

the story of the misteriously filling up hard disk

Napisane przez: koen roggemans ()
Liczba odpowiedzi: 20
Obraz Core developers Obraz Documentation writers Obraz Moodle HQ Obraz Particularly helpful Moodlers Obraz Plugin developers Obraz Translators

I just want to share my learning experience for future reference - it might help some others too puszcza oczko
We have a Moodle 2.1.2 with automatic backups enabled, once a week on friday, keep 1. After the upgrade in august from 1.9 to 2.1 we used about 400 GB from the 1000 GB disk for the moodledata folder. Last week, Nagios warned me we went over 80% disk usage, this week we were over 90%.
After a search, it appeared that the 'keep 1' setting didn't do the trick and I had all the backups since 20 august for all courses in the automated backup area. Why?
I found the cause in the 'Automated backup storage' setting. That was set to 'Specified directory for automated backups'. The 'Save to' setting was left black, which is ok, since the help message says 'Full path to the directory where you want to save the backup files
(leave blank to save in its course default dir)'. Therefore the backups were nicely saved in the 'Course backup file area', but it also caused the deleting of old backups not working anymore. Apparently if you set a 'Specified directory for automated backups', the automatic removing doesn't work.
I was happy to find that by running the cli script automated_backups.php the number of files was shrinking now to the expexted 1 in my course backup area. But the disk size kept on increasing. So not solved.
Deleted files apparently move to moodledata/trashdir. That contained now over 400GB. The trash dir is emptied accourding to the code, every 4 days. I didn't want to wait for that. In the database table mdl_config is a record 'fileslastcleanup', that says when the cleanup has last run (in time since epoch). I changed that time stamp, using http://www.epochconverter.com/ to a time stamp 4 days ago, started admin/cron.php. The trashdir was emptied and I'm back on 402GB disk usage.
Conclusion: be carefull with the 'Automated backup storage setting' puszcza oczko

Średnia ocen:Useful (25)
W odpowiedzi na koen roggemans

Re: the story of the mysteriously filling up hard disk

Napisane przez: James Rudd ()

Thanks for the great write up. I was unaware of the automated_backups.php script and that has simplified a problem I was having with runaway backups.

W odpowiedzi na koen roggemans

Re: the story of the misteriously filling up hard disk

Napisane przez: Rosario Carcò ()

I think this should be reported as a bug in the Tracker. Autmatic deletion of old backup-zip-files should work in any circumstance.

Rosario

W odpowiedzi na koen roggemans

Re: the story of the misteriously filling up hard disk

Napisane przez: Chris Kenniburg ()
Obraz Plugin developers

Could there have been a permissions issue with the "Specified Directory" that was preventing the deletion of old backups?  When we used a specified directory it had to be changed for the www-data user to have proper permissions to write and modify documents.

Just a thought on the issue.

W odpowiedzi na Chris Kenniburg

Re: the story of the misteriously filling up hard disk

Napisane przez: Melinda Kraft ()

It's not a permissions problem, or even a code one. It appears to be a hard coded setting that retains deleted fiels in the the Moodledata trash for 4 days. In a perfect world, I think Moodle admins should be allowed to set the trash clearing time to suit their particular needs, otherwise files should delete immediately.  In the case of backups, the problem can expand expontially if you retain a number of backups.

W odpowiedzi na Melinda Kraft

Re: the story of the misteriously filling up hard disk

Napisane przez: Ray Lawrence ()

Watch out for this too: MDL-29262 Massive database tables.

W odpowiedzi na Ray Lawrence

Re: the story of the misteriously filling up hard disk

Napisane przez: Ray Lawrence ()

Created an issue here: MDL-32100

W odpowiedzi na Ray Lawrence

Re: the story of the misteriously filling up hard disk

Napisane przez: koen roggemans ()
Obraz Core developers Obraz Documentation writers Obraz Moodle HQ Obraz Particularly helpful Moodlers Obraz Plugin developers Obraz Translators

Thank you for the rapport and adding me as a watcher Ray. Very thoughtfull of you.

W odpowiedzi na koen roggemans

Re: the story of the misteriously filling up hard disk

Napisane przez: Tony Holt ()

Just had this exact issue.

 

I'm okay to wait for the 4 days to be up, but our actual usage after moving the files to trash: 50GB
usage as a result of backups : 968GB.

 

Lesson Learned uśmiech

W odpowiedzi na koen roggemans

Re: the story of the misteriously filling up hard disk

Napisane przez: Nadav Kavalerchik ()
Obraz Core developers Obraz Plugin developers Obraz Testers Obraz Translators
W odpowiedzi na Nadav Kavalerchik

Re: the story of the misteriously filling up hard disk

Napisane przez: Rosario Carcò ()

Thanks, my data disk was filled more than once, when Moodle 1.9 did not correctly remove the existing zip-files. I just voted for it.

Rosario

W odpowiedzi na koen roggemans

Re: the story of the misteriously filling up hard disk

Napisane przez: Janine Hempy ()

You are my new best friend! Thanks for posting a detailed description of the problem and a solution. You rock!

W odpowiedzi na koen roggemans

Re: the story of the misteriously filling up hard disk

Napisane przez: Miki Alliel ()
Obraz Translators

Hey, we  use Moodle 2.3.3

and we set the "specified directory for automated backups" and set a valid Path. the backups are stored in the path (outside from moodledata) successfully but I can still see lot's of files in the list of the Automated bacuckup, I looked at the "moodledata/trashdir" and it is empty.  but still having this list of files in the  Automated bacuckup in the restore page of each course.

how can I delete them? our "moodledata/filedir" is huge now!

please Help

thanks

W odpowiedzi na Miki Alliel

Re: the story of the misteriously filling up hard disk

Napisane przez: David Andrew ()

This is happening to me too.

My Filedir is growing by 6Gig every night, I've already doubled the disk allocation once.

any help please

W odpowiedzi na David Andrew

Ynt: Re: the story of the misteriously filling up hard disk

Napisane przez: Erdem Sahin ()

Hi all,

I have the same problem with filedir. I need to find a way to decrease the size of filedir.

Is there a way to eliminate unnecessary files in filedir directory?

W odpowiedzi na Erdem Sahin

Re: Ynt: Re: the story of the misteriously filling up hard disk

Napisane przez: Chris Swaffield ()

I, too, would like suggestions on how to solve this huge fieldir problem. 

W odpowiedzi na Chris Swaffield

Re: Ynt: Re: the story of the misteriously filling up hard disk

Napisane przez: Bill Antonia ()

Have a look at my comments towards the end of this thread:

https://moodle.org/mod/forum/discuss.php?d=219826

I had the same issue, loads of backups despite limiting the number to keep. It's a little drastic as you have to alter the PHP code temporarily. However I gained around 180GB if I remember correctly.

Just make sure you have a good backup of your server first.

Regards

Bill