Thousands of .mbz files

Re: Thousands of .mbz files

by Ken Task -
Number of replies: 6
Picture of Particularly helpful Moodlers

There is a php script in moodlecode/admin/cli/ called 'automated_backups.php'.  Is that being run from a cron job or script outside of Moodle?   Even from another server as of 3.4 crons can be run from another server.

Is this a load balanced setup with nodes that have replicated code and thus each node server could call the scripts in code/admin/cli/?

'spirit of sharing', Ken



In reply to Ken Task

Re: Thousands of .mbz files

by Albert Ramsbottom -

Hi

Yes it is load balanced, so I am going to have a look now for multiple crons. However, I read on these forums that lots of MBZ files in temp/backup could be due to the recycle bin issue. I believe you have been involved in some of these threads

I have also been getting cron errors related to the recycle bin and this issue

Adhoc task failed: core_course\task\course_delete_modules,Coding error detected, it must be fixed by a programmer: The course module 1418453 could not be deleted. #0 /var/www/html/moodle/backup/util/factories/backup_factory.class.php(65):$

#1 /var/www/html/moodle/backup/controller/backup_controller.class.php(109): backup_factory::get_logger_chain(false, 1, 'b6eb5b047736a6a...')

#2 /var/www/html/moodle/admin/tool/recyclebin/classes/course_bin.php(123): backup_controller->__construct('activity', '1418453', 'moodle2', false, 10, '2')

#3 /var/www/html/moodle/admin/tool/recyclebin/lib.php(151): tool_recyclebin\course_bin->store_item(Object(stdClass))

#4 /var/www/html/moodle/course/lib.php(1185): tool_recyclebin_pre_course_module_delete(Object(stdClass))

#5 /var/www/html/moodle/course/classes/task/course_delete_modules.php(66): course_delete_module('1418453')

#6 /var/www/html/moodle/lib/cronlib.php(181): core_course\task\course_delete_modules->execute()

#7 /var/www/html/moodle/lib/cronlib.php(74): cron_run_inner_adhoc_task(Object(core_course\task\course_delete_modules))

#8 /var/www/html/moodle/admin/cli/cron.php(61): cron_run()

#9 {main}

Backtrace:

* line 181 of /lib/cronlib.php: call to core_course\task\course_delete_modules->execute()

* line 74 of /lib/cronlib.php: call to cron_run_inner_adhoc_task()

* line 61 of /admin/cli/cron.php: call to cron_run()

Cheers

In reply to Albert Ramsbottom

Re: Thousands of .mbz files

by Albert Ramsbottom -

Been checking my mdl_backup_controllers table as I have been getting .log files in the temp folder with info about backup controllers as well and these suggest backups running

I have 500 controllers listed, are these failures or successful? Should they clear out of the table?


Cheers

In reply to Albert Ramsbottom

Re: Thousands of .mbz files

by Albert Ramsbottom -

I have now got 500 or so backup controllers with a status of 700 (unfinished), that ran between Friday and Sunday. It seems that something was stuck in a loop and as we have got automated course backups turned off I think it maybe something to do with the recycle bin

All the backup controller entries have an itemID of 3878176. This I would have thought was something that has been recycled, maybe an activity

How can I find out what this itemID is and what course it lives in??

Cheers

In reply to Albert Ramsbottom

Re: Thousands of .mbz files

by Ken Task -
Picture of Particularly helpful Moodlers

The course re-cycle bin actually uses the backup process to create a small .mbz.

Working on a server right now that has a space issue and in investigating what was taking up an increase in disk usage ran this query ...

select filearea,userid,filename,filesize  from mdl_files where filename like '%.mbz'

example of what you will/should/might see:

recyclebin_coursecat    2    backup.mbz    5835
recyclebin_coursecat    2    backup.mbz    7590
recyclebin_coursecat    2    backup.mbz    165623892
recyclebin_coursecat    2    backup.mbz    5505995
recyclebin_coursecat    2    backup.mbz    5502692
recyclebin_coursecat    2    backup.mbz    5805
recyclebin_coursecat    2    backup.mbz    5673
recyclebin_course    2    backup.mbz    19325907
recyclebin_course    2    backup.mbz    20392869
recyclebin_course    2    backup.mbz    47135526
recyclebin_course    2    backup.mbz    62411002
recyclebin_coursecat    2    backup.mbz    79996

IF one were to find the contenthash of these backup files, locate the real file in the sea of files in moodledata/filedir/, copy that file out to a test directory, then un-compress it, one might see an xml file in the extract that gives hints/clues.

Was just looking at this on server am working with and the item above:

recyclebin_course    2    backup.mbz    62411002

is a total course backup - with a moodle_backup.log and moodle_backup.xml

Wow!  Someone needing an immediate solution to free up space now has a lot more work to do! :\

'spirit of sharing', Ken



Average of ratings: Useful (1)
In reply to Ken Task

Re: Thousands of .mbz files

by Ken Task -
Picture of Particularly helpful Moodlers

Follow up for space crunch ...

There are some settings for recyclebin duration ... https://site/admin/settings.php?section=tool_recyclebin

Default duration is 1 week.   If one needs space ASAP, one could set duration more frequently and then run the task for emptying recyclebin via task - run now - or via running the CLI version of those task.

Wonder if that really sends files to trashdir ... guess I'll have to experiment and see. :\

'spirit of follow up', Ken


Average of ratings: Useful (1)
In reply to Ken Task

Re: Thousands of .mbz files

by Josiah Pease -

You can query the DB :


function mmm { sudo -u apache rm sitedata/filedir/${1:0:2}/${1:2:2}/$1 ; }

mysql moodle <<< "select contenthash,filename from mdl_files where mimetype = 'application/vnd.moodle.backup' and filearea = 'recyclebin_course'" | while read line ; do mmm $line ; done


but these things are probably best done via moodle