[3.5]Thousands of info_from_mbz_.... files

Re: [3.5]Thousands of info_from_mbz_.... files

by Ken Task -
Number of replies: 11
Picture of Particularly helpful Moodlers

I take it the email you get about automated backups confirms that number and that there were no errors or pendings, etc.

368 where as originally reported was 3thousand3xx courses.

A bunch of courses must be hidden and according to auto bu settings, skip hidden courses excluded them.

Query your Moodle db courses table for id,fullname,shortname + visible=1 ... then again for visible=0 to get a count of visible vs in-visible.

Is the 368 accurate?

Hmmmm ... thought we were dealing with a Linux server, but see \temp\backup ... that a habit in typing or was this a Windows server all along?   Just askin' ... really don' t wanna know. smile

'spirit of sharing', Ken


Average of ratings: -
In reply to Ken Task

Re: [3.5]Thousands of info_from_mbz_.... files

by Richard van Iwaarden -
Picture of Particularly helpful Moodlers

Hi Ken,

The hidden courses were not excluded, I am doing so now. See also the answer from Séverin above.

We are dealing with a Linux server. I'm working on a several international keyboard layouts so sometimes I can't find the correct slash smile


I'm keeping an eye on the backup process... do you know when an email with report is being send out by Moodle? I haven't gotten one for quit some time now.


Average of ratings: -
In reply to Richard van Iwaarden

Re: [3.5]Thousands of info_from_mbz_.... files

by Ken Task -
Picture of Particularly helpful Moodlers

Check Task list??

'spirit of sharing', Ken

Average of ratings: -
In reply to Ken Task

Re: [3.5]Thousands of info_from_mbz_.... files

by Richard van Iwaarden -
Picture of Particularly helpful Moodlers

Where can I find that?

Average of ratings: -
In reply to Richard van Iwaarden

Re: [3.5]Thousands of info_from_mbz_.... files

by Ken Task -
Picture of Particularly helpful Moodlers

Site Admin Menu

Search box

task [ENTER]

Scheduled Task


Average of ratings: -
In reply to Ken Task

Re: [3.5]Thousands of info_from_mbz_.... files

by Richard van Iwaarden -
Picture of Particularly helpful Moodlers

That I do know, the scheduled task list is OK at my Moodle and shows no errors.

When you said 'task list' I thought that somehow I could see which tasks weren currently running.

Average of ratings: -
In reply to Richard van Iwaarden

Re: [3.5]Thousands of info_from_mbz_.... files

by Ken Task -
Picture of Particularly helpful Moodlers

?????

Back when attempting to get control of automated backups, it was suggested turning off the task for automated backups.  Truncate all the tables related. etc. etc.  After one successful run then check/turn on the scheduled task for automatted backups again.   Thought maybe you hadn't done that.

Cron job -> has individual task -> task are scheduled to run at certain times (or disabled/etc). 

Once completed they no longer show as any process anywhere - unless a run-away/zombie process.   And the only way of which I know where one could possibly see that is via command line on Linux - ps command with filters.

The only other way outside of the Moodle admin interface, is to setup cron job to also log to a log file.   One can then review the last log and/or watch that log file in real time as it executes.

Ken




Average of ratings: -
In reply to Richard van Iwaarden

Re: [3.5]Thousands of info_from_mbz_.... files

by Ken Task -
Picture of Particularly helpful Moodlers

Thinking on that some more ... question for you to research ... not everyone runs the same stuff in the same way.

How many of the task in the task list do you have activated and running?

Not sure I have a list ... not sure I wanna list either.

Guess one  has to rely upon the columns  related to when they last ran and if they completed successfully or not.

Don't think there is any single task that runs all the time without ever ending although it might kinda look like that ... if one follows the recommendation of running cron every minute.

Ken


Average of ratings: -
In reply to Ken Task

Re: [3.5]Thousands of info_from_mbz_.... files

by Richard van Iwaarden -
Picture of Particularly helpful Moodlers

We are running cron every minute and the automated backup is now out of the Moodle cron and started separately. I followed the exact copy of Serverin.

I could not follow your advise on that because you used Moosh, which I can not use.

My idea was that the past summer I started the automated backup and then it kept running for one month or so until it was finished. I understand that this is not the case now. It probably just fails at some point and is aborted. I have not seen one single Email from the system after August 6th... yet our automated backup starts on day 0, 2 and 4 of the week (see post above)

If I look at reports -> backup, I see that about 500 courses say 'automated backup pending'.


Yes, I did clean all tables once and did delete everything in Moodledata/temp/backup but this makes no difference, proces just restarts and also quits at some point leaving not much of an error log. Except for the reports->backup and the specific course it says something like:

Status - Unfinished
8:51:15 AM - setting controller status to 100
8:51:15 AM - loading controller plan
8:51:15 AM - instantiating backup controller 8c344bbf25a581564ecf291ccd69ab4f
8:51:16 AM - setting controller status to 300
8:51:16 AM - applying plan defaults
8:51:16 AM - setting controller status to 400
8:51:16 AM - setting file inclusion to 1
8:51:16 AM - checking plan security
8:51:16 AM - setting controller status to 700
8:51:16 AM - saving controller to db
8:51:16 AM - calculating controller checksum 13b1e3042ccf8a4da471f1baaeec37ce
8:51:16 AM - loading controller from db
8:51:16 AM - setting controller status to 700
8:51:16 AM - saving controller to db
8:51:16 AM - calculating controller checksum e6ef5e8971a7168ee2268e8bd7f8c181
8:51:16 AM - loading controller from db
8:51:16 AM - setting controller status to 800

All in all it still seems a BIG black box for me. It's a task I run and somehow it ends, but does not complete. Ever.

Average of ratings: -
In reply to Richard van Iwaarden

Re: [3.5]Thousands of info_from_mbz_.... files

by Ken Task -
Picture of Particularly helpful Moodlers

Server(s) are Debian's ... right? ... cannot or will not use moosh?  Don't answer that, you've already stated your preferences for GUI only - 80's or not.

"I followed the exact copy of Serverin."

Séverin's postings and quoting from his postings:
https://moodle.org/mod/forum/discuss.php?d=373771#p1513009

#4 "i invoque the cli script with memory_limit=2048M (perhaps less would be sufficient)"

and in ...
https://moodle.org/mod/forum/discuss.php?d=373771#p1512016
"Concerning backups, i've configured them to use a dedicated specified folder, and use manual launch (not automatic), skip hidden courses, and skip non modified courses from 60 days.

Then, i use a dedicated cron job to only launch the automatic backup script (admin/cli/automated_backups.php), allowing a lot of memory for PHP, and unlimited time (really 36 000)."

Have we not really changed approach to this issue? .... and expecting different/better results?

None of us know how large the courses are.   Moodle really doesn't have a plugin to calculate the size of backups before attempting any backups by any means.

But moosh does ..

This for a course entitled:
214    Honor's Digital Interactive Media

Digital Interactive Media includes video, audio, photography.
If you look at the info provided ...

Course ID: 214
No of contexts: 232
Context by level:
    CONTEXT_MODULE (70):    221
    CONTEXT_BLOCK (80):    11
Context by module:
    assign (1):    112
    forum (9):    1
    resource (17):    79
    url (20):    29
Number of role capability overwrites: 0
Role capability overwrites by context:
Number of local role assignments: 0
Locally assigned roles by context:
Number of enrolled users: 57
Users enrolled by role:
    editingteacher (3):    57
Number of groups: 2
Group statistics:
    Min number of members in a group:    24
    Max number of members in a group:    31
    Avg number of members in a group:    27
Course modinfo size: 423720
Number of sections: 12
Section statistics:
    Sections visible:    3
    Sections hidden:    9
    Min number of modules in a section:    5
    Max number of modules in a section:    54
    Avg number of modules in a section:    18
Number of grades: 6102
Number of log entries: 8512
Number of files: 4462
Total file size: 39783551842
Cache build time: 0.076740980148315

See the Total file size line?
39783551842 ... is over 300 Gig's.

On said server ... a K12 box ... I have 3 cron job'd bash shell scripts.  One of them does courses such as above.

That server will *NEVER* run automated backups because there are 15 courses similar to above whose backups range from 26+ gig -> the highest 318Gig.

Until the suggested improvement that would allow selection or deselection of courses to include/exclude in automated backups comes along, the only alternative I see is CLI with bash shell scripts from moodlecode/admin/cli/ or better yet, with moosh.

'spirit of sharing', Ken


Average of ratings: Useful (1)
In reply to Ken Task

Re: [3.5]Thousands of info_from_mbz_.... files

by Richard van Iwaarden -
Picture of Particularly helpful Moodlers

Ken, I probably didn't give you enough information on my situation.

The server is a CentOs or something with a WHM/Cpanel installed. I can only get to Cpanel, nothing more. Installing Moosh is not an option. There's no CLI option.

What it does have, is the cron-option in Cpanel. So I can start the Moodle cron every minute from there. What Séverin told me is to have an extra cron call on automated_backups.php, which I can do from Cpanel.

So I can do this: "Concerning backups, i've configured them to use a dedicated specified folder, and use manual launch (not automatic), skip hidden courses, and skip non modified courses from 60 days.' . I can do this without having a CLI. And I have done so.

If I'm being unclear it could be a language barrier in place. You try explaining all this to me in Dutch and you will probably understand it's not that easy smile

But to resume:

  1. I have no CLI
  2. I can run automated backups outside of Moodle cron
  3. I am using a dedicated directory for backups.


Hope you are not annoyed by my posts smile Your help and info is really appreciated.

Average of ratings: -
In reply to Richard van Iwaarden

Re: [3.5]Thousands of info_from_mbz_.... files

by Ken Task -
Picture of Particularly helpful Moodlers

Wasn't really annoyed ... just thought we were trying the same thing but expecting different results.   Understand about language ... BTW, your written English is better than mine! smile

Ahhhh.... sooooo ... cPanel/WHM combo ... so server isn't yours.  Have had experience helping another op that had that setup with provider ... had a 'reseller' type account ... that's the Web Hosting Management (WHM) piece of the combo.

So is the CentOS server one that has been allocated by the WHM? ... it has another IP address.

Even those have ssh but it has to be turned on to be able to access the allocated host.

One thing about WHM discovered ... it controls some of the configuation of the servers that have been allocated by it.   Some options in WHM for an allocated server's PHP memory as an example, can't go over 128M, if I re-call correctly.   Could login to the allocated host via ssh, and edit the config file ... which had a warning at the top of the file saying not to edit ... as changes in WHM would override whatever setting made manually.

Once on an allocated CentOS server.BTW, yum (the package manager) was there and one could install things like moosh without upsetting WHM.   Could even write bash shell scripts to tackle backup issues.

Those types of servers, IMHO, not friendly enough for dynamic Moodle whose requirements for XYZ keep going upwards.

'spirit of sharing', Ken

Average of ratings: -