Cron on a separate server

Cron on a separate server

by Jai Gupta -
Number of replies: 14
Has someone here tried to isolate cron and create a dedicated cron server?

I don't even know if it will provide any reasonable benefit. At present we use following to run cron.
A loop replaces 'Domain Name' with actual domain name and this complete scripts runs for every Moodle site (at present ~1200). On an average it takes 40 minutes for cron to complete and after that cron starts running again.

We want (if possible) to isolate this cron job from web server to a cron server.

1) Do you think it will be of an advantage?
2) What are the ways we can shift load from cron to some other server?

As per my understanding, we could use a plain server with only php installed (without apache, mysql). There are two questions about this.

1) Does cron needs access to moodledata folder as well?
2) What is the approximate ratio of select vs. update/insert operation in cron? This will help us judge if should use another database in master-master mode or we should use master db itself.

If there are many write operations in cron and we choose to use master db then the only load we will be shifting is php parsor load. How huge is this load in a web server?
Average of ratings: -
In reply to Jai Gupta

Re: Cron on a separate server

by Visvanath Ratnaweera -
Picture of Particularly helpful Moodlers Picture of Translators
Hi

I didn't get the idea. Apart from the scheduling priority adjustments what you do is to start a _local_ process

$ /path/to/php /path/to/moodle/admin/cron.php

It is not apparant to me, how you intend to kick cron.php on remote servers.

The usual trick to kick cron.php remotely is to request it through a http-client, for example:

$ wget http://server/path/to/cron.php

Sure you cat put thousends of wget calls in the crontab of a dedicated *cron-server". I don't know if it what you want.
Average of ratings: -
In reply to Visvanath Ratnaweera

Re: Cron on a separate server

by Jai Gupta -
Visvanath, thanks for your reply.

Yes, at present cron is a local process. My goal is to move the work (load) of cron to a different server.

If I run cron remotely using wget then also the load Moodle cron produces will stay on the existing server.
Average of ratings: -
In reply to Jai Gupta

Re: Cron on a separate server

by Visvanath Ratnaweera -
Picture of Particularly helpful Moodlers Picture of Translators
Hi

Still 99.9% of work will be within the Moodle server (or servers if web- and database-servers are different). In a normal setup at least. In your original posting you said something which needs an explanation:

> this complete scripts runs for every Moodle site (at present ~1200). On an average it takes 40 minutes for cron to complete and after that cron starts running again.

So these 1200 sites are in the same server?
Average of ratings: -
In reply to Visvanath Ratnaweera

Re: Cron on a separate server

by Jai Gupta -
Yes, around 1200 sites are on the same web server, we are using single code base. Moodle code is stored in a small ram drive which is accessed by all sites, storage for moodledata is provided with external storage. Each site has its own databases which are not part of web server.

All these 1200 sites have total of around 17K users, 4K courses, 26K resources and 30K questions (they are very small sites individually).

At present we don't run cron in parallel, everything is ok except that it takes around 40 minutes for cron task to complete. If we could run cron on different servers then it will become possible to run cron in parallel which would decrease cron interval. Our target is to be able to run cron at every 10 minutes.
Average of ratings: -
In reply to Jai Gupta

Re: Cron on a separate server

by Visvanath Ratnaweera -
Picture of Particularly helpful Moodlers Picture of Translators
Hi

There is a misunderstanding. As I wrote in my earlier posting, 99.9% of the work is done by the php process and its subprocesses. The cron daemon itself is the remaining 0.1%.

Therefore shifting the crond to another server won't have any noticeable effect. The php process is bound to the webserver, I don't see how you are going to shift that to a remote server.
Average of ratings: Useful (1)
In reply to Jai Gupta

Re: Cron on a separate server

by Nguyen An Thuan -
Why don't use a external cron jobs service, like www.setcronjob.com, webbasedcron.com, getcron.com, webcron.org, etc?
Average of ratings: -
In reply to Nguyen An Thuan

Re: Cron on a separate server

by Jai Gupta -
Any external cron job service will just call moodle/admin/cron.php but ultimately cron.php script will run on our own servers and hence load generated by cron will also stay on our own servers.

Goal is to move or shift cron.php script load from main web and DB server to some other server.
Average of ratings: -
In reply to Jai Gupta

Odp: Re: Cron on a separate server

by Bartosz Cisek -
Cron.php needs access to DB and moodledata files. Moving cron load to separate server you would need shared moodledata (maybe nfs) and copy of Moodle code with config.php for every instance.
Average of ratings: Useful (1)
In reply to Bartosz Cisek

Re: Odp: Re: Cron on a separate server

by Jai Gupta -
What all cron.php do with moodledata?

I can think of only automated backups.
Average of ratings: -
In reply to Jai Gupta

Odp: Re: Odp: Re: Cron on a separate server

by Bartosz Cisek -
Yep, I was thinking about auto backups.

Are there any other conditions under which cron.php touches moodledata?
Average of ratings: -
In reply to Bartosz Cisek

Re: Odp: Re: Odp: Re: Cron on a separate server

by Jai Gupta -
temp directory? I have no idea.
Average of ratings: -
In reply to Jai Gupta

Odp: Re: Odp: Re: Odp: Re: Cron on a separate server

by Bartosz Cisek -
Neither me. Do some tests smile
Average of ratings: -
In reply to Bartosz Cisek

Re: Odp: Re: Odp: Re: Odp: Re: Cron on a separate server

by Jai Gupta -
Finally our cron setup seems to working fine.

One DB server (replication)
Two cron servers with php and nfs (moodledata). Each server running 2 parallel cron task.

This has reduced average cron time from 40 minutes to 10 minutes.

It was interesting to experience that if we run cron more frequently average cron time reduces significantly.
Average of ratings: -