We run into the same problem near the end of each semester. When we have over 100 users all doing quizzes at once the system grinds to a halt. Page loads of over 30 seconds and frustrated staff and students. Any advice on the way forward would be great.
Rob, what is your current setup in terms of hardware?
Hi Simon + Rob (I assume you work together and Simon doesn't just magically know the answer there ;)),
Could we have some further information, including:
- what version of Moodle you're using;
- what PHP version;
- which PHP accelerator you're using;
- how much memory you've given that accelerator;
- what tuning you've done on your database server;
- what kind of filesystem backend you're using for your VM infrastructure.
If you can additionally look at tools like top, dstat, vmstat, etc that would also help. I personally prefer dstat:
dstat -Tmcndls
Best wishes,
Andrew
Hi Andrew,
Yes we do work together,
here is some of the information
what version of Moodle you're using; Currently running 2.4.6+
what PHP version; PHP Version 5.3.3-1
which PHP accelerator you're using; eAccelerator 0.9.6.1
how much memory you've given that accelerator; 16,777,144 Bytes
what tuning you've done on your database server;
database settings
skip-external-locking
key_buffer = 16M
max_allowed_packet = 16M
thread_stack = 192K
thread_cache_size = 8
myisam-recover = BACKUP
max_connections = 500
table_cache = 512
query_cache_limit = 1M
query_cache_size = 36M
long_query_time = 4
innodb_buffer_pool_size = 6144M
max_allowed_packet = 16M
[isamchk]
key_buffer = 16M
what kind of filesystem backend you're using for your VM infrastructure.
our VMware setup consist or two hosts connected to SAN
thanks
regards,
Simon
Your load average is extremely high
I processor should have a constant maximum of 1.0, OK all apache servers will peak and go to 3 or 4 but yours is showing 12.65
This suggests that you need more processors or cores to cope with the quiz when it happens. Not sure what environment you are using but if you are virtualised you can ask for more processor cores when a quiz is scheduled to take place
A high load average doesn't always mean that the root cause of a system's poor performance is a lack of CPU resources. I'm not saying this is or isn't the case here, but just that we can't know for sure either way based on just the load average data point.
Looks like it in this case!
Have a look at iostat, hdparm or sar whilst this is happening, should give some useful info
Cheers
Re: Does this data point to where our bottleneck is.
Just to correct a common misconception, the load average is the number of items waiting to be processed over a 1, 5, and 15 minutes period (IIRC). Therefore load average is tied directly to the number of processor cores you have available. In this case 2, so anything over 2 suggests that there's a bottleneck.
It does not suggest that you need more cores, but that there is a bottleneck which is causing delays. Throwing more processor cores at the issue could make it worse. For example, if this is an IO issue, then throwing more cores could increase IO and further slow the machine down.
Andrew
Re: Does this data point to where our bottleneck is.
Yes no misconception here, which is why I suggested using some more advanced tools such as IOstat, hdparm and SAR
A quick look at these tools one should be able to determine where the bottleneck is, I bet 10 quid if another processor is chucked at it, it will make a huge difference, after all this is a quiz which are processor intensive.
Albert
Do you have shell access to your web server that is running Moodle? If you do, can you run the command `vmstat 5 30` on the server while you are seeing performance problems? Post the output of the command here. It will give us a better understanding of what is happening on the server.