Moodle runs slow or crashes every day at the same time.

Moodle runs slow or crashes every day at the same time.

by Nathan Seaton -
Number of replies: 2

Everyday around 10:53am Moodle will typically slow to a crawl. Occasionally (a few times a month) Moodle will crash altogether. Initially I thought, "Ah! Must be a cron job."  However I stopped all cron jobs for 24 hours and saw the same issue. I've watched both machines and the database server seems to never have an issue however the processor usage on the Moodle server skyrockets to 100% right before the server stops responding. When I look at the current processes running on the Moodle server, nothing is using a ton of resources.

  • Virtualized with WMware vSphere
  • Moodle and Database are on seperate virtual servers
  • CentOS 5.6 x64 (both servers)
  • Background information:
  • Public School (Grades 5-12)
  • 2273 students active in past 18 days (1 laptop per student)
  • 278 teachers active in past 18 days (1 laptop per teacher)
  • Moodle 2.1.2
  • MySQL  5.0.77
  • PHP 5.3.3 
  • 65,829,987 db queries in the past 15 days
  • Moodle server
    • 4 virtual CPU's
      • Intel Xeon Core 2 @ 2.67GHz
    • 8GB of RAM
  • Database server
    • 1 virtual CPU
      • Intel Xeon Core 2 @ 2.67GHz
    • 4GB of RAM

I don't have any PHP accelerators installed at the moment. I've read through tons of threads, and tried tweaking some settings here and there like enabling dbsessions but see no improvement. Please tell me I'm overlooking something simple.

Thanks in advance,
Nathan 

Average of ratings: -
In reply to Nathan Seaton

Re: Moodle runs slow or crashes every day at the same time.

by Andrew Lyons -
Picture of Core developers Picture of Moodle HQ Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers Picture of Testers

Hi Nathan,

Much as I love to preach about how wonderful php accelerators are, the lack of one shouldn't cause an issue like this. That said, I'd highly recommend installing apc and tuning it. I've made a few responses in this forum recently with notes on how to tune it.

With the CPU sky-rocketting to 100% before it stops responding it could be a range of things. It would be helpful to find out what's causing the increase. What kind of CPU load is it - i.e. are you in IO Wait, or system, or user, or ...

A tool such as top would be able to tell you. I find that running in colour, showing all cores, full arguments listing, and only showing non-idle processes most helpful. On debian that's (open top), press z, 1, c, i. CentOS may differ in it's implementation of top, or at least it's default configuration of it. You could also have a play with iostat -v 1 to find out where your resources are going.

You say that you're virtualised using vSphere, one of our customers is using a similar setup and we've seen no issues, but the issue you'll find with virtualisation technologies such as vSphere is that resource sharing can have an indirect affect on other virtual servers. You say that there are no cron jobs running on the virtual servers, but have you checked the virtualisation layer? I suspect that you're in disk IO, and this is why your CPU jumps to 100%. This could be caused by a range of things...

As an example, I would assume that your disk architecture for these machines is probably backed off via vSphere to a SAN of some kind. It might be worth having a look at the latency graphs in vSphere for the disks. Is something happening at the time it freezes - e.g. backup? Is a virtual server snapshotting at the time that you run everything, or even the SAN?

If you could have a look at those areas you may find some useful pointers...

Hope that this helps,

Andrew

LUNS Limited

Average of ratings: Useful (1)
In reply to Andrew Lyons

Re: Moodle runs slow or crashes every day at the same time.

by Andrew Lyons -
Picture of Core developers Picture of Moodle HQ Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers Picture of Testers
Hi Nathan,

I was just wondering how you're getting on with your performance issues?

All the best,

Andrew
Average of ratings: -