General developer forum

Multiple cronjobs

 
Picture of Mark Sharp
Multiple cronjobs
Group Core developersGroup Particularly helpful MoodlersGroup Plugin developers

I've found myself with a queue of over 195k adhoc tasks in the mdl_task_adhoc table.

The cronjob is getting stuck slowly processing the queue to the detriment of all other adhoc and scheduled tasks.

The way I currently run cronjobs is to allow only one instance at a time - I create a lock file.

I will change that so that multiple crons will be able to run, but am concerned that I'm just going to load up apache with loads of cronjobs stuck on my enormous list of adhoc tasks.

I'm thinking of using timeout on the cronjob just to kill it and restart again

*/5 * * * * timeout 600 php /var/www/html/admin/cli/cron.php

Has anyone tried this? Any issues with doing this?

 
Average of ratings: -
Picture of Neill Magill
Re: Multiple cronjobs
Group Core developersGroup Plugin developers

I would not recommend terminating cron like that. You could end up with some cron tasks that are permanently locked (if you use file system locking)

This would mean that they would never run again, at least until you clean out your lock files. If your cron uses database locking then they should clear automatically after 24 hours though.

 
Average of ratings: Useful (1)
Me!
Re: Multiple cronjobs
Group Core developersGroup Moodle HQGroup Particularly helpful MoodlersGroup Plugin developersGroup Testers

"The way I currently run cronjobs is to allow only one instance at a time - I create a lock file."

And so you have broken the scalability of the cron system. 

It's designed to process scheduled and adhoc tasks in parallel. If you don't allow to do this, it simply won't keep up. 


 
Average of ratings: -