Cron Job questions

Cron Job questions

by Melissa Benson -
Number of replies: 19

Moodle 2.8.5+

Hopefully this is the right place. I have a few questions about cron jobs. All of the searching I've done just tells me how to set it up and doesn't really answer my questions.


  1. How many cron jobs are run (is it just one cron job with several tasks?)
  2. Is "scheduled tasks" the cron job? What is done during the moodle cron job?
  3. Are there best practices for when tasks are run?
  4. Could a cron job slow down the site?


I'm asking because I feel like the cron job keeps taking longer and longer. And I'm thinking it might be slowing down our site.

Average of ratings: -
In reply to Melissa Benson

Re: Cron Job questions

by Usman Asar -
Picture of Plugin developers Picture of Testers

1: yes 1 cron job takes on several tasks within Moodle.

2: Yeap, Cron Jobs is for Unix/Linux as Scheduled tasks is for Windows, and here are some of the tasks that it does

  • Legacy log table cleanup
  • Log table cleanup
  • Forum Mailing and maintenance jobs
  • Clean old sessions
  • delete unconfirmed and incomplete users
  • remove expire cache entries
  • clean backup table logs
  • send failed notifications
  • send self-enrolment expiration notifications
  • completion data, calendar notifications and so on..............

if you want, I can send you output of a CRON JOB to see for yourself.

 

3: tasks run in background, unless you are executing manually (where you can see the output). but you have option of getting the output sent to email as well. There is no best practice so to speak, where Moodle runs its Cron tasks every 30 mins, depending on how busy your site is, you can do hourly or even every 2, 3 or 4 hours.

4: yeap, it does slows down the site, if your site has lot of interaction, then keep cron jobs frequent, as if on a busy site you run cron job once a  day, it will then drink away your CPU and I/O's of the system, whilst running a CRON frequently, it will only take care of the data to be processed since last cron run.

Average of ratings: Useful (4)
In reply to Usman Asar

Re: Cron Job questions

by Melissa Benson -

Wonderful, thank you! A few more questions.


Maybe I'm thinking of the cron job differently. I was thinking it was a one time thing over night or something. Maybe because I'm seeing an email notification for backups and I thought backups were a a part of the cron job.

Are all of the tasks run every single time? If it's every minute..what happens when it takes longer than 1 minute and it overlaps?

My server guy said that a cron job was taking a long time and then we stopped it. How long should it take?


In reply to Melissa Benson

Re: Cron Job questions

by Usman Asar -
Picture of Plugin developers Picture of Testers

It's definitely not one time thing, consider this, the Moodle activity loads up the cache in Moodle Data Folders, and Cron flushes away that cache refreshing it until next cron job runs.

Yes all the tasks run in every cron job, and then it depends how much work has to be done on one particular task, and then cron takes longer or shorter. Just to give example, I have Moodle running with low traffic, and I run mine every 8 hours, now if I see the output of Cron that ran after 8 hours, it may have taken like 8-14 seconds (and the timings depend upon the activity that had been going around on my moodle since last Cron), and if I run Cron immediately after one cron has just taken place, 2nd one would finish in less than a second, as there was no activity in last one minute of difference.

Now taking another example, say for your institute, assuming it runs 8 A.M to 6 P.M with moderate traffic going on all day, and if you run cron every hour, it may take same time for completion on hourly basis (+/- few seconds), and in between these hours, lets say during lunch hour there is almost no activity, then the cron will finish instantly as compared to time taken in working hours.

As per your server guy, if the cron is taking long, means your cron jobs weren't set to be frequent, where Howard mentioned newer Moodle's recommending every minute, that may not be required for your institute, you have to do testings based on traffic and activity, like I said, you may have time when there is no activity and then times when there is huge traffic, so you can always adjust cron to run in a way, that during traffic hours it runs frequently (say every 30 mins or less) and in off hours, you can even set it to run every 3 hours or more. I seriously doubt that a Cron set to run every minute will take longer than a minute unless there is a scheduled data backup at one point then it can take slightly longer in that particular minute. 

Just for the sake of your understanding the Cron (and tasks it performs) I have taken output of a test Moodle site, I haven't set any automated crons running as its a test site with no activity other than mine, so a manual cron was run (1stcron.txt - took 12 seconds to complete) file attached, and immediately I took another manual cron within 2 minutes and it performed all the tasks that were done by first cron and yet took 0.7 seconds (2ndcron.txt attached), because there was nothing to take care of since last cron run 2 minutes before.

Have a look at the files, and you'll have idea of what CRON actually does and time difference between 2 cron jobs 2 minutes apart.

Average of ratings: Useful (3)
In reply to Melissa Benson

Re: Cron Job questions

by Usman Asar -
Picture of Plugin developers Picture of Testers

and if as per  your server guy, they have to shut down cron because it was taking too much time, then tell them to handle the cron in bursts until it gets streamlined, by that I mean run the cron and they can shut down if its taking too much time, then run again after few minutes, what that will do is, as last cron was partially completed, so next cron will (though starts all tasks again) but the first  few ones will be completed quickly, so in quick bursts they can complete the cron, and once completed fully they can safely schedule it to appropriate timings (depending on your traffic).

Cron jobs are CPU and Memory hog (even more if your database is running on same server), so for as long as its running, it will kill the performance of your site.

Average of ratings: Useful (3)
In reply to Melissa Benson

Re: Cron Job questions

by Michael Penney -
The cron runs on a schedule (often ever 15 minutes) but not every task runs every time, Some tasks like course backups might take a long time and sometimes on an overloaded server or when a bad choice has been made about how often to run the task, a cron will need to be killed.


For example, if this Administration > Site administration > > Courses > Backups > Automated backup setup. is suddenly set to run and it has never run before, it can take a long time. Or if it's not been set to backup logs and someone turns that on. Plug-ins can register different frequency depending on what they do. For example, if you look at course backups, you can set it to run at 2AM. How does it know when to start? The cron running every 15 minutes will regularly check to see what is scheduled to run. If the cron then runs at 2:14 and sees that it was supposed to start a backup job at 2:00, it will start the backup. So the cron might be running every 15 min, but the course backup will not run every 15 min, only when the next cron runs on or after the time the backup is set to run.

On the other hand, most sites want forum posts to go out as soon as possible, so usually the cron sends these out every time (if the cron hasn't been run in a while and you run it, it make take a long time as the forum posts get sent out, etc.).


Basically the cron is what checks in on various site things that need to happen regularly and will run the ones that need to run based on settings. If you set a task to happen every day at 1AM, the cron is what is going to cause that task to run. So for example you set a task to run at 3:00AM and your sys admin sets the cron to run every day at 1:00AM, your task will always be a day late smile.


Make sure your server guy didn't stop cron altogether, just killed the particular long running process, or lots of things you expect to work will stop working and when you do run it again, it may take a long time.

Average of ratings: Useful (5)
In reply to Melissa Benson

Re: Cron Job questions

by Andrew Lyons -
Picture of Core developers Picture of Moodle HQ Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers Picture of Testers

Hi Melissa,

Apologies in advance for the length of this response - I thought it would be helpful to answer in detail with a bit more information on some of the topics.

The cron system is there to run periodic tasks within Moodle. As others have noted, these include things such as course backups, statistics, log file cleanup, forum notifications, awarding badges, various cleanups, sending passwords to new users, etc. You can see a list of these tasks by going to Site administration -> Server -> Scheduled Tasks. Note, this list is mostly complete but some tasks are run by the "Legacy cron processing for plugins" task.

It has always been the case that we do not run all cron tasks every time you run cron. Some of these only happen once an hour, some once a day, some at pre-defined times of day, and some do run every time that you call cron.

In Moodle 2.7 we rewrote the cron system and renamed it the `Scheduled Task` system. The new system gives you, the administrator, much greater control over the way that cron is run on your system. For example, you can disable tasks or change their frequency from the user interface. Unfortunately it does not yet list the status of tasks but it does list the last time that a job was run, as well as the next time that it is due to start.

This new Scheduled Task system differs to the old Moodle cron system of 2.6 and before in two key ways. Firstly it breaks everything into more manageable parts. Rather than having a single cron task managed by a developer, it is now compartmentalised into a number of tasks which are individually controllable. The developer suggests sensible defaults which you can tune to your requirements. Secondly (in developer speak) it handles `locking` of tasks -- this means that it prevents the same task from running twice at the same time. This is the part that your server guy was concerned about - since Moodle 2.7, he no longer needs to worry about this.

Prior to Moodle 2.7, it was not advisable to have cron running multiple times simultaneously - in fact, it was to be avoided because it could cause problems.

Since Moodle 2.7, it doesn't matter if cron runs overlap. If this does happen, the second cron run just picks up the next task that was to be completed and the two processes help one another. This is great if you have   slow task waiting to complete. For example, the badge awarding process on moodle.org can take a very long time to complete. Before Moodle 2.7 when this task ran and we awarded the Particularly Helpful Moodler badge, forum notifications stopped being sent out. Since Moodle 2.7, the awarding of badges can still take a very long time, but it no longer prevents the forum notifications from being sent out.

Nowadays we recommend that you run cron as often as possible - if possible every minute. If you do not then there is an effect on end-user experience for two primary reasons:

  1. Notifications won't be sent! This is the main one. Things like forum notifications and new user passwords are sent by cron. If you only run cron once a day then your new users won't receive their passwords until the following day; and
  2. Several of the jobs are cleanup jobs. The remove old log entries, clear caches, and that kind of thing. Performance shouldn't really suffer too much with these in place (we build to scale), but it's generally better to have them running.

Regarding time to complete, I'm afraid that there is no fixed guideline. It really depends on many things, including:

  • hardware, database tuning, etc.;
  • number of active users;
  • number of badge awards;
  • number of forum posts being notified;
  • whether you're running course backups and how big those courses are;
  • whether you're processing statistics;
  • any enrolment plugins you have which need to synchronise;
  • how long the piece of string you keep in your back pocket is.

If you run cron less often then it will take longer to run. The forum notifications job can take a long time to complete if you have a lot of activity. If you run this more frequently then you have fewer messages to send each run. If you run it only once a day then you have to send every forum message for that day.

Generally it should never be necessary to interrupt cron. Doing so could mean that a job does not complete. In the case of forum notifications this would mean that some users do not receive notifications about new forum posts. The scheduled task system has timeouts in place to cull tasks which are taking longer than expected. Some jobs just take a long time. However this should not matter - it shouldn't generally affect your end-user performance.

Hope that this tome is of some assistance,

Andrew

Average of ratings: Useful (10)
In reply to Andrew Lyons

Re: Cron Job questions

by Andrew Lyons -
Picture of Core developers Picture of Moodle HQ Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers Picture of Testers

I forgot to mention about the impact that cron has on your server performance.

This entirely depends on your setup, including memory, web server, PHP version, how you can cron, which PHP cache you use, database + database version, CPU type, file system type, MUC configuration, number of users, time of day, phase of the moon, etc.

When I worked for Lancaster University we had a dedicated cron processing server (we had a lot 20,000 users, ran cron every minute, and had a number of complex custom tasks running to handle synchronisation of enrolments. If you have a larger institution then this may a route you later want to go down but it isn't generally necessary for most setups.

There will always be a hit on your performance when cron is running, but it should be negligible. Running cron more often should mean that you don't deal with such large datasets so you use less memory and it runs for less time.

Unfortunately there are too many factors to say what the exact effect will be.

Andrew

Average of ratings: Useful (5)
In reply to Andrew Lyons

Re: Cron Job questions

by Paul Lindgreen -
Picture of Particularly helpful Moodlers

Hi Andrew. Was is a huge job setting up a separate server for the cron jobs? if not, could you publish the steps? I guess a low spec'd server should be able to handle cron jobs if thats its only/main purpose?

We are running auto course backups for a few hours a day (and having some difficulties), it would be nice to get that process off the main web server.

Thanks

==

environment: windows 2008/iis7.5, moodle 2.7

In reply to Paul Lindgreen

Re: Cron Job questions

by Usman Asar -
Picture of Plugin developers Picture of Testers

Paul, how big are your courses?

how much resource allocation is given to your web server?

As I assist a school running IIS/MariaDB, on 2 cores vCPU 10GB RAM, and their course backups run absolutely fine.

In reply to Andrew Lyons

Re: Cron Job questions

by Jorge Villalon -

Hi Andrew,

We've been thinking on a separated cron server as well, we also have an incredibly active community in Moodle so the cron can spend hours sending email notifications (and everyone expects to get 1 email per message).

Do you know if the cron needs to access the moodledata folder or it could do its job just by having a connection to the database?

In reply to Andrew Lyons

Re: Cron Job questions

by Blair F. -
Picture of Particularly helpful Moodlers

This has been incredibly helpful, so I thank you!

I do have one question, though -- On our setup, the schedule showing under Scheduled Tasks for Automated Backups is every 50 minutes, even though on the Automated Backups settings page, we've set it to occur on Saturdays only, at 2am.  Why would they differ?  I've checked and it's running according to our settings and not those shown on the Scheduled Tasks page.

Thanks!

In reply to Andrew Lyons

Re: Cron Job questions-Legacy cron processing for plugins

by Paul Lindgreen -
Picture of Particularly helpful Moodlers
We are having some cron job issues, its hanging on a third party plugin, Blackboard collaborate. Their technical support suggest I change the frequency of the 'Legacy cron processing for plugins' task from every minute to once every 30minutes as a solution to the task hanging on their plugin.

Do you see any downside in their request? It seems to be working, our cron job isnt hanging on the blackboard elluminate task but i wasn't sure if the reduced frequency had any side effects.

We are on moodle 2.7.8.
In reply to Usman Asar

This forum post has been removed

The content of this forum post has been removed and can no longer be accessed.
In reply to Deleted user

Re: Cron Job questions

by Usman Asar -
Picture of Plugin developers Picture of Testers

Looks like you've a Virtual server and not actual hardware.

but anyway, https://docs.moodle.org/30/en/Cron_with_Windows_OS

Follow this link to set up your CRON

Average of ratings: Useful (1)
In reply to Usman Asar

This forum post has been removed

The content of this forum post has been removed and can no longer be accessed.
In reply to Usman Asar

This forum post has been removed

The content of this forum post has been removed and can no longer be accessed.
In reply to Melissa Benson

Re: Cron Job questions

by Howard Miller -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

The recommendation for Moodle 2.7 and newer is to run cron every minute. 

However, if you can'y or don't want to do this it does no harm to do it less frequently (just some things will get delayed). Just make sure you *do* run it though. 

Average of ratings: Useful (4)
In reply to Howard Miller

Re: Cron Job questions

by Mark Sharp -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Plugin developers

You'll also get more control over what gets run and when, when cron does run, through the config at  /admin/tool/task/scheduledtasks.php (2.8 defo, 2.7 not sure, =<2.6 not there).

Average of ratings: Useful (1)