Hopefully this is the right place. I have a few questions about cron jobs. All of the searching I've done just tells me how to set it up and doesn't really answer my questions.
- How many cron jobs are run (is it just one cron job with several tasks?)
- Is "scheduled tasks" the cron job? What is done during the moodle cron job?
- Are there best practices for when tasks are run?
- Could a cron job slow down the site?
I'm asking because I feel like the cron job keeps taking longer and longer. And I'm thinking it might be slowing down our site.
1: yes 1 cron job takes on several tasks within Moodle.
2: Yeap, Cron Jobs is for Unix/Linux as Scheduled tasks is for Windows, and here are some of the tasks that it does
- Legacy log table cleanup
- Log table cleanup
- Forum Mailing and maintenance jobs
- Clean old sessions
- delete unconfirmed and incomplete users
- remove expire cache entries
- clean backup table logs
- send failed notifications
- send self-enrolment expiration notifications
- completion data, calendar notifications and so on..............
if you want, I can send you output of a CRON JOB to see for yourself.
3: tasks run in background, unless you are executing manually (where you can see the output). but you have option of getting the output sent to email as well. There is no best practice so to speak, where Moodle runs its Cron tasks every 30 mins, depending on how busy your site is, you can do hourly or even every 2, 3 or 4 hours.
4: yeap, it does slows down the site, if your site has lot of interaction, then keep cron jobs frequent, as if on a busy site you run cron job once a day, it will then drink away your CPU and I/O's of the system, whilst running a CRON frequently, it will only take care of the data to be processed since last cron run.
Wonderful, thank you! A few more questions.
Maybe I'm thinking of the cron job differently. I was thinking it was a one time thing over night or something. Maybe because I'm seeing an email notification for backups and I thought backups were a a part of the cron job.
Are all of the tasks run every single time? If it's every minute..what happens when it takes longer than 1 minute and it overlaps?
My server guy said that a cron job was taking a long time and then we stopped it. How long should it take?
It's definitely not one time thing, consider this, the Moodle activity loads up the cache in Moodle Data Folders, and Cron flushes away that cache refreshing it until next cron job runs.
Yes all the tasks run in every cron job, and then it depends how much work has to be done on one particular task, and then cron takes longer or shorter. Just to give example, I have Moodle running with low traffic, and I run mine every 8 hours, now if I see the output of Cron that ran after 8 hours, it may have taken like 8-14 seconds (and the timings depend upon the activity that had been going around on my moodle since last Cron), and if I run Cron immediately after one cron has just taken place, 2nd one would finish in less than a second, as there was no activity in last one minute of difference.
Now taking another example, say for your institute, assuming it runs 8 A.M to 6 P.M with moderate traffic going on all day, and if you run cron every hour, it may take same time for completion on hourly basis (+/- few seconds), and in between these hours, lets say during lunch hour there is almost no activity, then the cron will finish instantly as compared to time taken in working hours.
As per your server guy, if the cron is taking long, means your cron jobs weren't set to be frequent, where Howard mentioned newer Moodle's recommending every minute, that may not be required for your institute, you have to do testings based on traffic and activity, like I said, you may have time when there is no activity and then times when there is huge traffic, so you can always adjust cron to run in a way, that during traffic hours it runs frequently (say every 30 mins or less) and in off hours, you can even set it to run every 3 hours or more. I seriously doubt that a Cron set to run every minute will take longer than a minute unless there is a scheduled data backup at one point then it can take slightly longer in that particular minute.
Just for the sake of your understanding the Cron (and tasks it performs) I have taken output of a test Moodle site, I haven't set any automated crons running as its a test site with no activity other than mine, so a manual cron was run (1stcron.txt - took 12 seconds to complete) file attached, and immediately I took another manual cron within 2 minutes and it performed all the tasks that were done by first cron and yet took 0.7 seconds (2ndcron.txt attached), because there was nothing to take care of since last cron run 2 minutes before.
Have a look at the files, and you'll have idea of what CRON actually does and time difference between 2 cron jobs 2 minutes apart.
and if as per your server guy, they have to shut down cron because it was taking too much time, then tell them to handle the cron in bursts until it gets streamlined, by that I mean run the cron and they can shut down if its taking too much time, then run again after few minutes, what that will do is, as last cron was partially completed, so next cron will (though starts all tasks again) but the first few ones will be completed quickly, so in quick bursts they can complete the cron, and once completed fully they can safely schedule it to appropriate timings (depending on your traffic).
Cron jobs are CPU and Memory hog (even more if your database is running on same server), so for as long as its running, it will kill the performance of your site.
For example, if this Administration > Site administration > > Courses > Backups > Automated backup setup. is suddenly set to run and it has never run before, it can take a long time. Or if it's not been set to backup logs and someone turns that on. Plug-ins can register different frequency depending on what they do. For example, if you look at course backups, you can set it to run at 2AM. How does it know when to start? The cron running every 15 minutes will regularly check to see what is scheduled to run. If the cron then runs at 2:14 and sees that it was supposed to start a backup job at 2:00, it will start the backup. So the cron might be running every 15 min, but the course backup will not run every 15 min, only when the next cron runs on or after the time the backup is set to run.
On the other hand, most sites want forum posts to go out as soon as possible, so usually the cron sends these out every time (if the cron hasn't been run in a while and you run it, it make take a long time as the forum posts get sent out, etc.).
Basically the cron is what checks in on various site things that need to happen regularly and will run the ones that need to run based on settings. If you set a task to happen every day at 1AM, the cron is what is going to cause that task to run. So for example you set a task to run at 3:00AM and your sys admin sets the cron to run every day at 1:00AM, your task will always be a day late .
Make sure your server guy didn't stop cron altogether, just killed the particular long running process, or lots of things you expect to work will stop working and when you do run it again, it may take a long time.
Apologies in advance for the length of this response - I thought it would be helpful to answer in detail with a bit more information on some of the topics.
The cron system is there to run periodic tasks within Moodle. As others have noted, these include things such as course backups, statistics, log file cleanup, forum notifications, awarding badges, various cleanups, sending passwords to new users, etc. You can see a list of these tasks by going to Site administration -> Server -> Scheduled Tasks. Note, this list is mostly complete but some tasks are run by the "Legacy cron processing for plugins" task.
It has always been the case that we do not run all cron tasks every time you run cron. Some of these only happen once an hour, some once a day, some at pre-defined times of day, and some do run every time that you call cron.
In Moodle 2.7 we rewrote the cron system and renamed it the `Scheduled Task` system. The new system gives you, the administrator, much greater control over the way that cron is run on your system. For example, you can disable tasks or change their frequency from the user interface. Unfortunately it does not yet list the status of tasks but it does list the last time that a job was run, as well as the next time that it is due to start.
This new Scheduled Task system differs to the old Moodle cron system of 2.6 and before in two key ways. Firstly it breaks everything into more manageable parts. Rather than having a single cron task managed by a developer, it is now compartmentalised into a number of tasks which are individually controllable. The developer suggests sensible defaults which you can tune to your requirements. Secondly (in developer speak) it handles `locking` of tasks -- this means that it prevents the same task from running twice at the same time. This is the part that your server guy was concerned about - since Moodle 2.7, he no longer needs to worry about this.
Prior to Moodle 2.7, it was not advisable to have cron running multiple times simultaneously - in fact, it was to be avoided because it could cause problems.
Since Moodle 2.7, it doesn't matter if cron runs overlap. If this does happen, the second cron run just picks up the next task that was to be completed and the two processes help one another. This is great if you have slow task waiting to complete. For example, the badge awarding process on moodle.org can take a very long time to complete. Before Moodle 2.7 when this task ran and we awarded the Particularly Helpful Moodler badge, forum notifications stopped being sent out. Since Moodle 2.7, the awarding of badges can still take a very long time, but it no longer prevents the forum notifications from being sent out.
Nowadays we recommend that you run cron as often as possible - if possible every minute. If you do not then there is an effect on end-user experience for two primary reasons:
- Notifications won't be sent! This is the main one. Things like forum notifications and new user passwords are sent by cron. If you only run cron once a day then your new users won't receive their passwords until the following day; and
- Several of the jobs are cleanup jobs. The remove old log entries, clear caches, and that kind of thing. Performance shouldn't really suffer too much with these in place (we build to scale), but it's generally better to have them running.
Regarding time to complete, I'm afraid that there is no fixed guideline. It really depends on many things, including:
- hardware, database tuning, etc.;
- number of active users;
- number of badge awards;
- number of forum posts being notified;
- whether you're running course backups and how big those courses are;
- whether you're processing statistics;
- any enrolment plugins you have which need to synchronise;
- how long the piece of string you keep in your back pocket is.
If you run cron less often then it will take longer to run. The forum notifications job can take a long time to complete if you have a lot of activity. If you run this more frequently then you have fewer messages to send each run. If you run it only once a day then you have to send every forum message for that day.
Generally it should never be necessary to interrupt cron. Doing so could mean that a job does not complete. In the case of forum notifications this would mean that some users do not receive notifications about new forum posts. The scheduled task system has timeouts in place to cull tasks which are taking longer than expected. Some jobs just take a long time. However this should not matter - it shouldn't generally affect your end-user performance.
Hope that this tome is of some assistance,
I forgot to mention about the impact that cron has on your server performance.
This entirely depends on your setup, including memory, web server, PHP version, how you can cron, which PHP cache you use, database + database version, CPU type, file system type, MUC configuration, number of users, time of day, phase of the moon, etc.
When I worked for Lancaster University we had a dedicated cron processing server (we had a lot 20,000 users, ran cron every minute, and had a number of complex custom tasks running to handle synchronisation of enrolments. If you have a larger institution then this may a route you later want to go down but it isn't generally necessary for most setups.
There will always be a hit on your performance when cron is running, but it should be negligible. Running cron more often should mean that you don't deal with such large datasets so you use less memory and it runs for less time.
Unfortunately there are too many factors to say what the exact effect will be.
Hi Andrew. Was is a huge job setting up a separate server for the cron jobs? if not, could you publish the steps? I guess a low spec'd server should be able to handle cron jobs if thats its only/main purpose?
We are running auto course backups for a few hours a day (and having some difficulties), it would be nice to get that process off the main web server.
environment: windows 2008/iis7.5, moodle 2.7
We've been thinking on a separated cron server as well, we also have an incredibly active community in Moodle so the cron can spend hours sending email notifications (and everyone expects to get 1 email per message).
Do you know if the cron needs to access the moodledata folder or it could do its job just by having a connection to the database?
This has been incredibly helpful, so I thank you!
I do have one question, though -- On our setup, the schedule showing under Scheduled Tasks for Automated Backups is every 50 minutes, even though on the Automated Backups settings page, we've set it to occur on Saturdays only, at 2am. Why would they differ? I've checked and it's running according to our settings and not those shown on the Scheduled Tasks page.
Do you see any downside in their request? It seems to be working, our cron job isnt hanging on the blackboard elluminate task but i wasn't sure if the reduced frequency had any side effects.
We are on moodle 2.7.8.
Thanks for the detailed and useful response. I need moodle (installed version is 3.1.1 with php 5.6.25 on windows server 2012 with IIS 8 please refer the attached image for machine specifications) to send the forum post notifications to students/course participants.
I hope Cron will help me in this regard. I have searched setup cron job on windows machine, this is the link http://gyretechnologies.com/Blog/Setup-Moodle-Cron-with-Windows-Task-Scheduler that guides in setting up cron job.
Any thoughts/suggestions would be more than welcome.
Thanks for you time and efforts
Thank you usman. I will update you.