Hardware and performance

Hi Gary

Quote
> One thing I had been doing just before the crash was to switch back and forth several times between the Bubble and Compact theme to see if this really caused all previous lines of chat to disappear or if it was just my machine.

That is highly suspicious. The theme is part of Moodle code and it has to be clean. If possible fall back to a core theme. BTW, do you have theme designer mode ON?

> After the 4th or 5th switch in rapid succession I noticed I couldn't send a line I'd been typing and the two others reported by email they lost me and each other in chat along with access not only to Moodle but to the entire domain. And when I tried to go in remotely to the server I noticed I couldn't and a physical reboot of the server needed to be done to get going again.

That is absurd. If 10 web visitors can take down the whole bunch of servers of an institution, there is something wrong in a big way. Are all these hosted on one (physical) server? Are you sure of the hardware, device drivers? A serious stress testing is needed from the part of the system administrators.

Will spare all the long story about 2 Elementary classrooms and 6 chat rooms on a 2 Gig server, but will say only that 2 Gig's was not enough memory to handle that number of chat rooms.  

Chat is a separate little daemon to itself and in Moodle, chat can be configured to use actual chat daemon software, not the built in.   In my investigation of that, each chat room used approximately 512K memory.   So my experience was with a 2 Gig box, but even with 4 Gig a single chat room might push a server over limits depending upon other daemons ... like apache/mysql/etc..

Reason for top ... known to be installed on all CentOS systems and easiest to get a quick look at memory usage/activity on a box un-seen ... lots of detail left out of the first posting.   Yes, plenty of other/better monitoring tools, but then op might have issues installing one of those those ... plus there is proper config of a monitor ... ever configure a monitor to consume more of the server than intended thus creating your own DOS?

4 Gig is normally a good place to start but it's been my experience that through time and usage of Moodle, a stand alone (apache/mysqld/other running on same server) running Moodle will eventually consume more memory ... period.

@Howard as also said that other things worth looking at ... my experience with similar incidents (one with a wiki - which was sort of a DOS) confirms that one could still login (via ssh) and issue commands although extremely slow.   Apache still running.  Server still up (if you pinged it) ... didn't crash.   As far as core dumps and .core files, think the default behavior for CentOS is to save the .core file in the directory of the app that caused the issue.  Can be configured differently.  And, yes, one could find other daemons on the server causing core dumps so looking at the entire server was a good idea, but since OP was talking about chat in Moodle ... then that's why I suggested looking for .core dumps where I did.

@Gary ... well, new shells are discovered frequently.  Who's to say your server doesn't have one ... but if you followed advice given, one is about as 'secure' as one can humanly get and be able to use what's installed!   It always has been an 'element of risk' when running anything like a Moodle ... or a Joomla ... or a WordPress ... or a .... on forever.

For the benefit of others, the source referenced by Gary is an overall guide on Web Shells - Threat Awareness.   https://www.us-cert.gov/ncas/alerts/TA15-314A

Good thing to be aware.  What was advised on the reference I read makes sense for anyone running anything open source on port 80  - one recommendation ... keep your software up to date.  Duh!  Ya think?   

Moodle (like Joomla/WordPress/Drupal, etc. any well known + good open sourced software) is serious about security, but users of that software have to do their part ... keeping up to date.

Recently spent 2 solid days upgrading 1.5.x Joomlas to 3.highest on a server that had WP as well.  Joomla's were vulnerable to 'web shell' trojans via a plugin installed in the Joomla's.   The WP's on same server did suffer from that as the web shell trojans/header injections etc. from Joomla's spilled into them as well.

So @Gary ... while there are many approaches to this, I'd recommend a place to start is installing ClamAV on server and doing a scan of the entire Moodle code directory - anything at web root.  ClamAV will detect some things like trojans, etc.

Am sure @Howard and @Visvanath can share other tools, etc..

'spirit of sharing', Ken

Hi Visvanath, I did scan the whole system from / and found nothing. Will check out the link you provided. Many thanks.