Preventing Moodle to periodically delete backup directory?

Preventing Moodle to periodically delete backup directory?

by Nicolas Martignoni -
Number of replies: 12
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers Picture of Translators
Background

To provide the MoodleBox image (free as in speech!, see https://moodlebox.net/ and https://github.com/martignoni/moodlebox), I install Moodle on a Raspberry Pi 3. Main reasons for such a tiny server are: hardware price, mobility and access to a Moodle site when no Internet available. MoodleBox is widely used in Africa, South and North-America and in Europe.

For performance reasons, I moved the moodledata/temp on a RAM filesystem (tmpfs). But RPi RAM size is modest (1GB) and I need to keep moodledata/temp (typically around 50 MB). This small size is generally not a problem for the temp directory contents, except for backup files, which are written to moodledata/temp/backup (during backup/restore) and can be very big (typically more than 100 MB).

My solution to this is currently to soft-link the moodledata/temp/backup directory outside of moodledata/temp, e.g. to moodledata/backup (mkdir -p moodledata/backup; ln -s moodledata/backup moodledata/temp/backup), and then to install Moodle.

This seems to work. However, occasionally, the backup directory is deleted by Moodle, and this deletion breaks (evidently) the backup/restore process, since Moodle can't find moodledata/temp/backup, can't create it either (since we've a link in this place), and drops an error ("error/cannot_create_backup_temp_dir").

Questions

  1. Where (in the code) and when (and why) is Moodle deleting the moodledata/temp/backup directory? I looked everywhere in the code, without finding it. I can't figure out in which circumstances this occurs. I even tried every scheduled tasks without finding it.
  2. As a workaround, do you know any way to prevent Moodle to periodically delete the moodledata/temp/backup directory? I tried to change the umask and user and group permissions, but this didn't work.
  3. As another workaround, I think of re-creating periodically the moodledata/backup directory (via cron). Any thoughts about such a solution?
  4. Wouldn't it be a good idea for Moodle to have the possibility to store the backup files (sometimes very big!) outside of moodledata/temp, in a specific location in moodledata (or elsewhere), enabling more flexibility for the sysadmin? This could be a config.php setting, just like CFG->tempdir or $CFG->cachedir.

Any help and/or hint would be greatly appreciated.

Nicolas

Average of ratings: Useful (1)
In reply to Nicolas Martignoni

Re: Preventing Moodle to periodically delete backup directory?

by Tobias Reischmann -

Hey Nicolas,

the responsible task should be '\core\task\file_temp_cleanup_task'. This process periodically goes through all temp directories and checks their change date.

Another workaround, which comes to my mind, is to simple do not give your webserver user write permission to the backup temp dir. Looking at the code of the cleanup task, this should result in an error log. However, your link should remain were it is.

Cheers,

Tobias

Average of ratings: Useful (2)
In reply to Tobias Reischmann

Re: Preventing Moodle to periodically delete backup directory?

by Nicolas Martignoni -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers Picture of Translators

Thanks Tobias, I'll look into this.

Edit: I can reproduce the deletion now. But, even when the web server has no write permission, the backup dir is deleted. Have to dig deeper. smile

Average of ratings: -
In reply to Tobias Reischmann

Re: Preventing Moodle to periodically delete backup directory?

by sam marshall -
Picture of Core developers Picture of Peer reviewers Picture of Plugin developers
Hi Nicholas,

Based on Tobias's information, I suggest you create a cron job that simply touches the directory / a file in the directory once per day. The default tempdatafoldercleanup is apparently a week so that should prevent it from ever deleting it? Rather than recreating it after deleted, this seems safer. (If you can't get the permissions to work!)

--sam

Average of ratings: Useful (3)
In reply to sam marshall

Re: Preventing Moodle to periodically delete backup directory?

by Nicolas Martignoni -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers Picture of Translators

Sam and Tobias, you made my day smile

Sam's workaround works a treat. I'll adjust the cron down to once per hour, in case when the admin changes the setting "tempdatafoldercleanup" to its lower limit wink

Ciao!

Nicolas

Average of ratings: -
In reply to Nicolas Martignoni

Re: Preventing Moodle to periodically delete backup directory?

by AL Rachels -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers

Hi Nicolas,

Just a thought, if the memory on your Raspberry Pi 3 is a problem, you might want to try an Odroid-XU4 or Odroid-HC1. They both have 2GB of LPDDR3 RAM. Their 8 core CPU's and 2 GHz speed is a very noticeable improvement over the PI 3's. Price is very reasonable for either one.

If your server needs to have a monitor, the XU4 has HDMI. The HC1 is cheaper, no monitor connection, no moving parts, and a SATA connector for 2.5inch HHD/SSD. I currently have one that uses a 64GB MicroSDC for the Ubuntu 16.04 OS and a 500GB SSD for extra storage. (Videos from Red5-still working on getting this the way I want it.) Supposedly, Hardkernal is going to release a version that can accommodate 3.5inch drives.

Average of ratings: -
In reply to AL Rachels

Re: Preventing Moodle to periodically delete backup directory?

by Nicolas Martignoni -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers Picture of Translators

Hi Daniel,

Thanks for you input. I've already thought of RPi alternatives like the Odroid or OrangePi, but I'll stick with the Raspberry, since it's much cheaper than other similar hardware (though less powerful). For the audience of the MoodleBox, the price seems to be an important factor. The RPi is very well supported too, which simplifies the support.

As a matter of fact, memory (and speed) is not so much a problem, but I want to avoid to drain too much of it with a too big RAM partition. This is the reason to avoid backup files in the temp directory.

Anyway thanks again for your thoughts.

Nicolas

Average of ratings: -
In reply to Nicolas Martignoni

Re: Preventing Moodle to periodically delete backup directory?

by Marcus Green -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers

I have been impressed with the performance so far.

Average of ratings: -
In reply to Nicolas Martignoni

Re: Preventing Moodle to periodically delete backup directory?

by Séverin Terrier -
Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Testers Picture of Translators

Hi,

Glad to see you've found a solution for the MoodleBox smile

But i still think that being able to define in config.php something like $CFG->backuptempdir would be useful.

Perhaps it needs a tracker issue?

Séverin

Average of ratings: -
In reply to Séverin Terrier

Re: Preventing Moodle to periodically delete backup directory?

by Nicolas Martignoni -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers Picture of Translators

I find in too a good idea. But has it any chance to be implemented? What do you (and other) thinks?

Average of ratings: -
In reply to Séverin Terrier

Re: Preventing Moodle to periodically delete backup directory?

by Nicolas Martignoni -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers Picture of Translators

Just created MDL-60923. Please comment.

Average of ratings: Useful (1)