Hello, any news on this? I don't think I am the only one with this issue :/
Hi Christos,
Just wanted to reply to say you're not the only one experiencing this! We've had to turn off all course backups just to ensure all the other adhoc tasks continue to run. Forum notifications were being delayed by days and the teachers were very unhappy. I fiddled around with the database to move the next backup date to very far in the future to prevent them from backing up (as the adhoc tasks had already been set up and were proceeding regardless of me telling the task not to run!).
I'm currently trying to see if there's another way to run the automated backups without letting Moodle do it itself. I might have to build a script that does the backups using MOOSH. It's a real pain!
I'll update you if and when I get any further.
All the best,
Rich
Great news! After a lot of fiddling, reading through tracker tickets and testing on our development environments, I think I have a fix. It turns out there is undocumented functionality that allows you to specify how many of each task can run simultaneously. If you add the following to your config.php file, it will only allow one instance of the backup task to run at any time.
$CFG->task_concurrency_limit = array('core\task\course_backup_task'=>1);
If you combine this with the new task processing settings under: /admin/settings.php?section=taskprocessing , you can set "Scheduled task concurrency limit" and "Ad hoc task concurrency limit" to something higher than 1 (I've set 5) then it should only use 1 cron to run backups and the spare 4 to do all the other processing.
As I've got a large backlog of backups to run, I will fiddle around with the number of simultaneous backups so we can get some running in parallel - not too many though, to avoid thrashing the DB.
I found the information in this ticket, as well as a way of testing it, but it really ought to be documented in the main Moodle docs - or at least hinted at. https://tracker.moodle.org/browse/MDL-64610
$CFG->task_concurrency_limit = array('core\task\course_backup_task'=>1);
If you combine this with the new task processing settings under: /admin/settings.php?section=taskprocessing , you can set "Scheduled task concurrency limit" and "Ad hoc task concurrency limit" to something higher than 1 (I've set 5) then it should only use 1 cron to run backups and the spare 4 to do all the other processing.
As I've got a large backlog of backups to run, I will fiddle around with the number of simultaneous backups so we can get some running in parallel - not too many though, to avoid thrashing the DB.
I found the information in this ticket, as well as a way of testing it, but it really ought to be documented in the main Moodle docs - or at least hinted at. https://tracker.moodle.org/browse/MDL-64610
Thank you for the reply!
So, to understand this better
when you say that you have 5 crons you mean you have 5 servers running the Moodle cron or you start 5 instances of the cron at the same time?
If you mean 5 servers, this makes sense to me so 1 of the servers will be stuck running backups and other 4 will do other stuff.
If you mean 5 instances, could you please tell me how you achieve this?
Thank you once more for taking the time to share your finding Really appreciate it.
So, to understand this better
If you mean 5 servers, this makes sense to me
If you mean 5 instances, could you please tell me how you achieve this?
Thank you once more for taking the time to share your finding
Hi Christos,
We are experiencing the same issue. I think that what Richard meant was to configure the settings on http://[your_moodle_site]/admin/settings.php?section=taskprocessing to "Ad hoc task concurrency limit" = 5 and add the variable $CFG->task_concurrency_limit = array('core\task\course_backup_task'=>1); to your config.php....
I will try that!
Best,
susana
We are experiencing the same issue. I think that what Richard meant was to configure the settings on http://[your_moodle_site]/admin/settings.php?section=taskprocessing to "Ad hoc task concurrency limit" = 5 and add the variable $CFG->task_concurrency_limit = array('core\task\course_backup_task'=>1); to your config.php....
I will try that!
Best,
susana
Hi,
We were not successful setting the parameters described. We still have forum notifications delayed and of course complaints on our community. We had no issues with Moodle 3.5....
Does anyone know where we can find documentation on this new task management?
Thank you,
susana
We were not successful setting the parameters described. We still have forum notifications delayed and of course complaints on our community. We had no issues with Moodle 3.5....
Does anyone know where we can find documentation on this new task management?
Thank you,
susana
(Moodle 3.9 PostgreSQL)
I was running into this problem as well. In addition to setting the concurrency limits, I had to adjust the number of records that get_next_adhoc_task was fetching. There is a hard coded limit in lib/classes/task/manager.php of 2000 adhoc tasks to be examined, so if there are more than 2000 backups, other adhoc tasks like forum notifications won't be processed whether or not the concurrency limits would allow it. I've just submitted a bug report about this. MDL-72313
For the background see the discussion "Managing scheduled tasks & php load on server" https://moodle.org/mod/forum/discuss.php?d=417100, especially this post https://moodle.org/mod/forum/discuss.php?d=417100#p1680821.
Thanks so much for this! I've just come back to this thread after a year absence. I've now increased the query limit to 10,000. This explains why we were having no problems until our courses got busy at the start of term. We generally have about 7000 active courses on our Moodle site, so backing up these was filling up the 2000 limit on adhoc tasks, so Moodle couldn't see the ~14,000 forum notifications waiting to be sent!
I've changed the value and hopefully backups and other tasks will now be able to coexist happily. I'm considering putting in some logic into config.php which allows a larger number of concurrent backups to process outside of our core teaching hours. This would help the backups process a little more efficiently.
Thanks again!
I've changed the value and hopefully backups and other tasks will now be able to coexist happily. I'm considering putting in some logic into config.php which allows a larger number of concurrent backups to process outside of our core teaching hours. This would help the backups process a little more efficiently.
Thanks again!