Increase in disk space usage in 2.2 to 2.7 upgrade

Increase in disk space usage in 2.2 to 2.7 upgrade

by Walter Horowitz -
Number of replies: 7

Our school has been running Moodle 2.2 for over a year with Moodledata on a 72 GB disk volume  with less than 50% utilization. We are running on Ubuntu with Apache and Postgresql. We use a separate volume for Moodledata and the database storage. Over the summer, we upgraded to Moodle 2.7. After the upgrade, the disk usage jumped to 60% and a few days later the drive went to 100% utilization. I reconfigured the drives on the server so that the volume was 144 GB of disk space and of course, disk space utilization dropped to 50%. Disk space utilization has been steadily growing and now it is up to 80%.

The backups are directed to a separate volume, so that they shouldn't be the issue. The Moodle system has had almost no usage since it is summer here, so I am very concerned that as school starts, we will run out disk space again. Since the system ran all last year using less than 50% of a 72 GB volume, I don't understand why just upgrading to version 2.7 has made the system approach filling a 144 GB volume.

Average of ratings: -
In reply to Walter Horowitz

Re: Increase in disk space usage in 2.2 to 2.7 upgrade

by Dan Marsden -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers Picture of Plugins guardians Picture of Testers Picture of Translators

First it would be good to analyse where the large files are - is the temp folder filling up or is it the general file pool? - I've also seen other weird disk configuration errors where disk space is not cleared correctly when a file is actually removed. 

I'd make sure that your cron process is running correctly.

I'd also make sure that the user running apache has rights to delete files in your moodledata directory.

We have a coursesize report that could help to find some of the files if they are sitting within a particular course:

https://moodle.org/mod/forum/discuss.php?d=212532#p926137

Once you have determined where the files are coming from it should be easier to work out what is going wrong.

Average of ratings: -
In reply to Dan Marsden

Re: Increase in disk space usage in 2.2 to 2.7 upgrade

by Walter Horowitz -

Thanks for the response. The increase is in the filedir directory, an increase of 61 to 101 GB for a system that has effectively no usage. I am now looking at the filedir to see if I can figure out what files are being created, but it isn't easy.

Cron is working properly. At the moment, it looks like the system has stopped growing quickly, but school starts tomorrow.

Average of ratings: -
In reply to Walter Horowitz

Re: Increase in disk space usage in 2.2 to 2.7 upgrade

by Dan Marsden -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers Picture of Plugins guardians Picture of Testers Picture of Translators

have you tried the coursesize report I linked above? - it might help you diagnose it further.

as you've mentioned you use course backups I'd still suspect them as the cause - are you sure you don't have "Automated backup storage" set to course backup file area or both file area and specified path?



Average of ratings: -
In reply to Walter Horowitz

Re: Increase in disk space usage in 2.2 to 2.7 upgrade

by Ken Task -
Picture of Particularly helpful Moodlers

With school 'starting tomorrow' think that is normal to see an 'up-tick' and growth in filedir.   Might be really concerned IF there is no activity in trashdir, however.

To see what ... about the only way is via a DB query ... something like:

select contenthash,filearea,filename,filesize from `mdl_files` where (`filesize` not like '0') order by `filesize` desc

Gee, maybe this year, teachers have learned to backup their course once it's ready for the first day of school!!! ;)   Have any teachers that decided  to give 'flipped instruction' a try?

'spirit of sharing', Ken

Average of ratings: -
In reply to Walter Horowitz

Re: Increase in disk space usage in 2.2 to 2.7 upgrade

by Walter Horowitz -

Automated backups are set to go to /media/usbdrive/m2012 and the nightly backup makes sure the USB drive is mounted because sometimes it isn't after a system reboot. What happens to the Moodle internal automated backups if the specified directory isn't available? Our courses have automated backups that could explain the growth if backups were stored in the filedir area. Is there a reasonable way to know if the automated backups that Moodle lists are stored in the filedir area?

Average of ratings: -
In reply to Walter Horowitz

Re: Increase in disk space usage in 2.2 to 2.7 upgrade

by Dan Marsden -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers Picture of Plugins guardians Picture of Testers Picture of Translators

if the path listed isn't available I'm pretty sure Moodle stores it in the course backup area (filedir)

this should list files stored in filedir related to backups:

SELECT f.contextid, sum(f.filesize) as filessize
FROM mdl_files f
WHERE component like 'backup' GROUP BY f.contextid

you might need to match that against the context table to get the right course locations though.

Average of ratings: Useful (1)
In reply to Dan Marsden

Re: Increase in disk space usage in 2.2 to 2.7 upgrade

by Walter Horowitz -

Yes, the increase in disk space usage is from backups being stored in filedir instead of the backup path.  However, our system settings specify to keep 2 copies of each backup. The increase in storage is because older backups are not being deleted, either in filedir or the specified backup area. 

Is this a bug?

Average of ratings: -