What is the max allowable size for course backups?

What is the max allowable size for course backups?

by Jennifer Blandino -
Number of replies: 9

I work on a project that has implemented Moodle 3.11, and we currently have a course that is very large. Program management refuses to split the course in any way, so unless we can come to them with documentation from Moodle showing a max size for a course, we're stuck in our current situation.


I'm currently attempting to make a backup of the course, and the backup fails with the error "Cannot create local file pool file. Please verify permissions in dataroot and available disk space."


The attempt to backup did create a folder in the data/temp/backup directory, and the backup.mbz file is around 4.9 GB. This is an increase in size from our previous backup, which was at 2.5 GB last month. We have added a lot to that course since then.  I am no longer able to create a backup successfully via the UI anymore, as I get this error, and the backup file isn't available in the Course Backup Area, though I can find the mbz in the temporary folder, as I mentioned. I get the same error if I attempt to generate the backup via cli at the command line. I also get the same error if I attempt to upload the .mbz on the course's restore page.


My question is, is there a documented maximum size allowable for course backups, both in generation and in restoration? If so, what is it, and how can I find it? I need this information to take back to my program management to prove to them that the size of the course is the issue holding us back.


Also, how can we find out in Moodle whether a course is becoming "too big to back up"?

Average of ratings: -
In reply to Jennifer Blandino

Re: What is the max allowable size for course backups?

by Ken Task -
Picture of Particularly helpful Moodlers

"Cannot create local file pool file. Please verify permissions in dataroot and available disk space."

Where does moodledata reside?

Did you check free space where ever moodledata resides?

Did you check ownership/permissions on moodledata and on moodledata/filedir/ in particular?

So how much free space to you have.   The last step in the process is to 'copy' that backup.mbz so system really need at least 4.x Gig more space for that operation.

That backup.mbz file is a valid backup file can be moved to a file system repo and be seen in moodle.

Had  the same problem with a course backup as large as 130+ Gig but could only back it up via command line scripts in admin/cli/ and pointed to save to a moodle file system repo in moodledata/respository/largebackups/ directory. And set to a cron job that ran at non-prime time usage.

'SoS', Ken

Average of ratings: Useful (1)
In reply to Ken Task

Re: What is the max allowable size for course backups?

by Jennifer Blandino -
Is moodledata the directory that contains subdirectories like backups, cache, cronlogs, localcache, temp, sessions, etc.? If so, it resides directly under the server's C drive, and the permissions seem to be correct on it for both PHP and Moodle. We dismissed the possibility of its being a permissions issue at once because we've found that we're able to create backups of our other, smaller courses, but not this big one. The disk space on the drive is hundreds of gigabytes away from being full, so that isn't the problem either.

In php.ini we've set our post_max_size and upload_max_filesize values to 7000M in the hopes that these values have some bearing on the problem. However, that isn't the case, which is to be expected, since we're making a backup, not uploading a file.
Average of ratings: -
In reply to Jennifer Blandino

Re: What is the max allowable size for course backups?

by Ken Task -
Picture of Particularly helpful Moodlers

moodledata is the typically named directory for all uploads/caches, etc. for moodle.  Yours evidently c:\data\ ... so there's a hint/clue your forgot to mention ... operating system is Windows.    Is this an XXAMP install of Moodle?   or IIS?  What is version of PHP and php-cli and those a 64 bit version or a 32 bit version?

Tweaking the max sizes for uploads to 700M won't do anything for this issue ... not uploading.   However there are other PHP settings which might come into play ... time for a script to run, memory a script can consume.   Big process like a backup might extend beyond your settings there.

There is also potential for conflict of cron job cleanup task which are supposed to 'clean up' backup/temp/.   Cron job running 1 time every minute?   Are there other directories that contain backup.mbz files in them (as well as the xml file + related subdirectories?

'SoS', Ken

Average of ratings: -
In reply to Ken Task

Re: What is the max allowable size for course backups?

by Jennifer Blandino -
It's an XXAMP install, with version 7.4.14 (cli) PHP x64.

I wouldn't mind investigating those other php settings and seeing if they're giving us this heartburn. I'll try those, as well as whether or not we're colliding with the cron task, and update my post with the results. Thanks for y our help!
Average of ratings: -
In reply to Jennifer Blandino

Re: What is the max allowable size for course backups?

by Ken Task -
Picture of Particularly helpful Moodlers

Do check server error logs as well.

Uhhh ...

https://docs.moodle.org/311/en/Windows_installation_using_XAMPP

"XAMPP is a fully functional web server package. It is built to test web based programs on a personal computer. It is not meant for online access via the web on a production server."

"Be aware that XAMPP is designed for developers and so deliberately has just about all security features turned off."

And this troubled course .... how many SCORM modules does it have in it?  50+?

I used to (past tense) support 5 Corp Moodles - United Technologies back then .. now just Carrier Corp. and related industries ...  don't think I ever saw a course with 50+ SCORM modules in a single course.

Not at all sure that's 'sound' course design nor conducive to learning/training - if that's the goal of that course.

'SoS', Ken

Average of ratings: Useful (1)
In reply to Jennifer Blandino

Re: What is the max allowable size for course backups?

by Ken Task -
Picture of Particularly helpful Moodlers

Separate posting with some suggestions ...

One of the reasons I had that 130+ Gig course was course was not reset at close of academic session for 2 years in a row.   Does your course have an ending time/date/cutoff where it could be reset?

The other ... what's in the course that could be 'off-loaded' to some other location?   Example: lots of mp4 videos could be hosted elsewhere and just linked to ... only the link would be in the backup ... not the videos.

 'SoS', Ken

Average of ratings: -
In reply to Ken Task

Re: What is the max allowable size for course backups?

by Jennifer Blandino -
Our customers don't want the course reset. They want it open indefinitely. It has no ending time or cutoff.

The contents in this course are primarily SCORM lesson activities, and a bunch of them. Some 50 of them, to be exact. If there is a way to offload those, I'd love to hear it, because even when the backups are made correctly, it takes forever with so much overhead.
Average of ratings: -
In reply to Jennifer Blandino

Re: What is the max allowable size for course backups?

by Ken Task -
Picture of Particularly helpful Moodlers

One more question ... how long has this course been running?   Does 'client' forsee a day when this course will no longer be required?   When? 2030, 2040, 2050?

Is the expectation of 'client', that this 'forever course' should never loose users accounts, would always be able to view ALL data on ALL users?   If so, you'd better check other settings site wide - 'life' of a user account even if in-active for years ... as an example - other settings ... events/tracking/completions etc.

This is not really a question for discussion/justifications ... just a thought ... Uhhh ... site such as that will only grow ... and one doesn't have 'unlimited' space, nor processing power, nor unlimited x y z.

'SoS', Ken

Average of ratings: Useful (1)