Hi All,
I have our Moodle 1.8.2+ production environment running on a Server 2003 SP2/IIS 6 box, using PHP 5.1.2 and MySQL 5.0.27. Server specs are as follows -
2xAMD CPU (6GHz Total)
4GB RAM
Up until recently the site ran absolutely fine without any problems, then last Thursday the w3wp started to consume 60-80% CPU time while the site was idle and 100% whenever any accesses were made to our Moodle site. Nothing has changed on the server or to the moodle application in this time.
Our site is http://www.merthyrlearns.co.uk
Any help greatfully received!
Thanks,
Ceri
Windows server: Production Site Running Extremely Slow/IIS Consuming 100% CPU
by Ceri Richards -
Number of replies: 10
In reply to Ceri Richards
Re: Windows server: Production Site Running Extremely Slow/IIS Consuming 100% CPU
by Ken Wilson -
Hi Ceri
This is a long shot... but take a look at your cron output (load this in your browser by entering http://www.merthyrlearns.co.uk/admin/cron.php) and check for any processes that are still running. Prime suspects are either backups or statistics.
If it's the former, check if there has been a large upload into your moodledata folders (right click -> properties and look at the total file size). If course contents are getting large and causing problems like this, use the external zip and unzip programs (see this Moodle docs for the GNUWin32 Windows versions) to run your backups. If you're still getting problems, consider using an external backup solution, e.g. NTBackup, Backup Exec, etc to backup moodledata.
If statistics is a problem, then try turning it off
.
Let us know how it goes!
Thanks
Ken
This is a long shot... but take a look at your cron output (load this in your browser by entering http://www.merthyrlearns.co.uk/admin/cron.php) and check for any processes that are still running. Prime suspects are either backups or statistics.
If it's the former, check if there has been a large upload into your moodledata folders (right click -> properties and look at the total file size). If course contents are getting large and causing problems like this, use the external zip and unzip programs (see this Moodle docs for the GNUWin32 Windows versions) to run your backups. If you're still getting problems, consider using an external backup solution, e.g. NTBackup, Backup Exec, etc to backup moodledata.
If statistics is a problem, then try turning it off
. Let us know how it goes!
Thanks
Ken
In reply to Ken Wilson
Re: Windows server: Production Site Running Extremely Slow/IIS Consuming 100% CPU
by Ceri Richards -
Hi Ken!
First of all - you are a legend!!! The cron output revealed the cause of our problems. It looks as if the cron job somehow ran perpertually and continued to run during the day. I have no idea why this occurred as it has always run overnight in the past. Would it cause any issues if I download the most recent cron client and re-install?
My first step is definately to change to GNUWin32 zip, then to disable all course backups if this fails as we are covered with BackupExec anyway.
Thanks again both for your help! Any ideas why the cron job would cause this issue - I've set the service to re-start automatically as I've noticed it crashed out quite alot!
Ceri
First of all - you are a legend!!! The cron output revealed the cause of our problems. It looks as if the cron job somehow ran perpertually and continued to run during the day. I have no idea why this occurred as it has always run overnight in the past. Would it cause any issues if I download the most recent cron client and re-install?
My first step is definately to change to GNUWin32 zip, then to disable all course backups if this fails as we are covered with BackupExec anyway.
Thanks again both for your help! Any ideas why the cron job would cause this issue - I've set the service to re-start automatically as I've noticed it crashed out quite alot!
Ceri
In reply to Ceri Richards
Re: Windows server: Production Site Running Extremely Slow/IIS Consuming 100% CPU
by Ken Wilson -
Hi Ceri
Good to hear that you've found the problem. I don't think re-installing the cron client would change anything given that it's the backup process in the cron which is causing the problem. Best thing to do is to stop IIS, install those GNUWin32 utilities and set the system paths to point to them. Then restart IIS again.
It's probably a big course that's causing the problems, so take a look at the total size of your moodledata folder and decide what to do in terms of your backup strategy, keeping in mind that if you do use BackupExec then it backs-up the *whole site* - not the courses. So if someone wants their course restored, you're going to have to do it to a non-production server first then take the course off there.
All the best!
Ken
In reply to Ceri Richards
Re: Windows server: Production Site Running Extremely Slow/IIS Consuming 100% CPU
by Dan Marsden -
also - what are you using to run php? - standard cgi, isapi, or fastcgi?
I've seen some suspicious behaviour with fastcgi that has cleared itself with an iisreset (didn't diagnose it further as I don't look after moodle on windows anymore.....)
Dan
I've seen some suspicious behaviour with fastcgi that has cleared itself with an iisreset (didn't diagnose it further as I don't look after moodle on windows anymore.....)
Dan
In reply to Dan Marsden
Re: Windows server: Production Site Running Extremely Slow/IIS Consuming 100% CPU
by Ceri Richards -
In reply to Ceri Richards
Re: Windows server: Production Site Running Extremely Slow/IIS Consuming 100% CPU
by Dan Marsden -
actually, we saw some pretty big performance gains using FastCGI in IIS 6 over isapi- you just need to be very careful with timeout values, and play around with it in a test environment first before running in production.
(except I did notice some strange behaviour occaisionally....)
Dan
(except I did notice some strange behaviour occaisionally....)
Dan
In reply to Dan Marsden
Re: Windows server: Production Site Running Extremely Slow/IIS Consuming 100% CPU
by Ceri Richards -
Thanks Dan - slightly un-related, but I played with APC recently. This completely killed our development environment running PHP 5.1.2 and Moodle 1.8.2+.
Tried it on another environment running PHP 5.2.5 (whatever the latest is) and Moodle 1.9 and ran ok for the majority but did produce the odd error and blank page. Set the memory cache to 64MB initially and made sure this wasnt exceeded but still no joy!
Would you recommend any other PHP accelerators which run ok on an IIS box?
Thanks,
Ceri
Tried it on another environment running PHP 5.2.5 (whatever the latest is) and Moodle 1.9 and ran ok for the majority but did produce the odd error and blank page. Set the memory cache to 64MB initially and made sure this wasnt exceeded but still no joy!
Would you recommend any other PHP accelerators which run ok on an IIS box?
Thanks,
Ceri
In reply to Ceri Richards
Re: Windows server: Production Site Running Extremely Slow/IIS Consuming 100% CPU
by Dennis Flynn -
just my 2-cents given the versions of PHP I see in this post...
When I ran 5.1.2, w3.exe would consume 100% CPU till I restarted IIS. When I moved to 5.2.1, all problems went away.
I never moved to 5.2.5, because I saw other people post problems that seemed to indicate that what was happening in 5.1.2 had returned.
So, I'm a 5.2.1 zealot until proven otherwise (which I am fully willing to be proven otherwise!)
In reply to Dennis Flynn
Re: Windows server: Production Site Running Extremely Slow/IIS Consuming 100% CPU
by Ceri Richards -
Thanks Dennis - definately useful information.
Think I'll push for our sites to be upgraded to PHP 5.2.1 and Moodle 1.9+ during the summer!
I also gave up using a PHP accelerator in the end. Performance was better overall, but there were too many occurances of blank pages and page cannot be displayed errors. I also kept an eye on the memory size and followed the instructions to the letter!
I may revisit once we upgrade!
Cheers,
Ceri
Think I'll push for our sites to be upgraded to PHP 5.2.1 and Moodle 1.9+ during the summer!
I also gave up using a PHP accelerator in the end. Performance was better overall, but there were too many occurances of blank pages and page cannot be displayed errors. I also kept an eye on the memory size and followed the instructions to the letter!
I may revisit once we upgrade!
Cheers,
Ceri
In reply to Dan Marsden
Re: Windows server: Production Site Running Extremely Slow/IIS Consuming 100% CPU
by Luis de Vasconcelos -
Dan,
I've given in to temptation and will be setting up IIS 6 with FastCGI on a Windows 2003 Server SP2 test box to compare it to my current ISAPI setup. But first 2 questions from your post:
What "strange behaviour" did you get?
Can you post more details about what you meant by "be very careful with timeout values"?
Thanks