Crashing under increased activity... But not a lot ... 20 users maybe at the same time?

Crashing under increased activity... But not a lot ... 20 users maybe at the same time?

by Billy Zwiener -
Number of replies: 6

Can anyone guide me to where to look first at performance issues that inevitably cause a crash when around 20 users might be taking quizzes at the same time? I'm on a VPS with plenty of resources... Possibly I have something configured wrong? I'm looking for any ideas as to where to start troubleshooting . Thanks

Average of ratings: -
In reply to Billy Zwiener

Re: Crashing under increased activity... But not a lot ... 20 users maybe at the same time?

by Emma Richardson -
Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Plugin developers

What version of Moodle?  I used to have issues with quizzes on earlier versions but later versions are so much better performance wise.  I used to fix by throwing more RAM  and CPU power at the server and that would normally fix it.  Is you database on the same server or a different one?  You might look at that connection and check resources while the quiz is happening...

In reply to Billy Zwiener

Re: Crashing under increased activity... But not a lot ... 20 users maybe at the same time?

by Ken Task -
Picture of Particularly helpful Moodlers

How many questions on a single page of the quiz?   Reduce the number of questions per page.

Check servers logs ... especially mysql logs.   Quiz is pretty hefty hitting on DB.   If you see, 'Server has gone away' in error logs, it's max_packets_allowed for DB ... default is 16M ... add config and set to 32M.

'spirit of sharing', Ken

In reply to Ken Task

Re: Crashing under increased activity... But not a lot ... 20 users maybe at the same time?

by Billy Zwiener -

Moodle 2.9 Thanks. Ill check the mysql logs... Here is a lot more details if it might help.


The problems are  sporadic  with my moodle installs that have been very hard to find the issue. When there is a problem it affects the performance of the site running moodle (not 

the site that is static pages... meaning, its only affecting the performance of moodle) It usually starts with the CSS breaking and elements appearing in strange places because of that.

Sometimes, refreshing the page will put things back (making images reappear, etc) but sometimes the entire moodle install wont function or display. This will happen across every moodle install on the server. I have 4 running moodle installs on my VPS. They are test installs except one which is the primary one for students to login.

 

At one time I had a developer say that the php version was incompatible and he changed it, however in the moodle environmnet page it shows everything in perfect condition, and yet the problem still intermittently exists.


 

The temporary fix I have resorted to is restarting the virtual environment via the Power Panel


for the VPS. You can see in the top left how theres an option to restart. This seems to fix it, but it’s not a permanent fix and it will eventually start occurring again, with the page styles breaking and then the page becoming unreachable.

You can also see a red exclamation point showing a warning for the “container” yet when I look at the resources it always shows they are well under any max


 

 Again, when one install starts doing this they will all be acting the same. Because of this, I assume its a server issue BUT why does the moodle environment page always show things are good? Can you investigate a little to see what might be causing the issue?

 

As far as resources on my VPS, they SHOULD be plenty. I have use of 4 cores, 8000 MB memory, 3TB of bandwidth, and 240 Gigs of disk space.

 

I only have about 30 users a month on teh website so the load should not be so large??? I do have a few third party scripts running that are integrated with TWILIO that help me with my chat room, but again, the resources I have shouldn’t make it an issue.


You can see here that when I start to see elements not appearing on the page, it’s a sign that the page is going to crash soon… along with all the other moodle installs running on that VPS. They will all be displaying similar breaks. 



 

Problems are definitely occurring when the main login pages have no style sheet referenced any longer.


 

Which will lead into no display at all (and I say its crashed whether that’s accurate or not)


 

This last crash occurred 13:15:00 on 4/26/16 and it might have been coincidence but I was uploading and configuring SCORMS on a test site. Its also noteworthy when a lot of people are taking quizzes that it seems more prone to “crashing”. Again, this could be coincidence as when I looked immediately at power panel at my resources WHILE the moodle site was crashed and it showed 


 

(again, all my resources well below max YET the red container exclamation still warns “Some Virtual Environment Resources are overused”


Noteworthy, when I click to see the details it shows the resources above pictured (which shows everything running fine as far as I can see.

 

Again, all 4 installs behave exactly the same when theres a problem and all 4 installs are fixed after restarting the VPS environment in Power Panel. Those are my biggest clues to offer you. I would love to find resolve to this problem as I have been experiencing issues with this for months and hostgator support or anyone else can find the cause.

 

 


In reply to Billy Zwiener

Re: Crashing under increased activity... But not a lot ... 20 users maybe at the same time?

by Billy Zwiener -

Perhaps this is a clue as well. My dcachesize is on red alert during some time periods??? dcachesize

In reply to Billy Zwiener

Re: Crashing under increased activity... But not a lot ... 20 users maybe at the same time?

by Ken Task -
Picture of Particularly helpful Moodlers

Uhhh ... let's get some terms down so I understand ... 'crash', to me, means a core-dump.  Is that the same to you? or is that the moodle(s) and whatever else you are running are really slow and appear to be un-responsive?

I think you mean the latter.

Below are 'educated' guesses ....

Lots of cacheing going on with Moodle now-a-days and see you have opcache.   There are some tools, not part of Moodle code, which will enable you to check on the health of opcache, flush files, etc.  You get to pick your own poison:

http://www.stevejenkins.com/blog/2014/07/my-favorite-zend-opcache-status-scripts/

When themes go south like that could be cached issues.

Your server sounds pretty hefty and should NOT be suffering from lack of resources ... which then leads to config ... but what?     This is an all-in-one box ... DB server on same host.   How much memory has been allocated to the DB server?   I use a thing called mysqltuner.pl (a perl script) to check settings and performance of DB server.   I have noticed a site that is busy will need increases to settings in order to attempt to keep it performing well.

There is a setting for max connections ... defaut is 150 + 1 the +1 for admin user via cmmand line access - that I've had to tweak more than once on a server for a K12 ISD that has gone bring your own device.

Apache is running mod or fastcgi?

Got enough apaches and workees in reserve?   What does 'top' look like at a busy time?   What does top F+P to see what might be using swap space look like?   What does: ps aux |grep httpd (or web service) look like?

Running a GUI destkop on it with all sorts of applets running ... screen saver, trashlet, or something no one is really using?

Someone/something poking at probing at your box?   I also run multitail and set it up to watch not only apache logs (access and error) but also others ... like secure, messages, etc..  Occassionally find an IP address that's attempting to what appears to be a denial of service .... thousands of attempts in a short time frame.   That sort of thing makes the NIC's too busy, but the server and services are running just fine!  Love re-routing those off into la-la-land. ;)

So there's really all sorts of potential bottlenecks/issues ... just gotta keep investigating till ya find it.

'spirit of sharing', Ken

In reply to Ken Task

Re: Crashing under increased activity... But not a lot ... 20 users maybe at the same time?

by Billy Zwiener -

Thank you for the time to offer your insight, Ken. I really do appreciate it. Although my head spins with a lot of your tech lingo (and yes, you were correct with my misuse of "crash") I WILL be able to hand this off to other skilled techs who can help me thanks to you helping me. A lot of the techs sometimes are out of the loop of what moodle does so this will hopefully aid me in communicating with them... again, thank you very much for your wisdom. Take care.