Eslam, we are trying but hard to do remotely. None of us a capable of a 'Vulcan Mind Meld'!
We (ok, maybe just me) might send you on what appears to be incorrect paths but at least they eliminate possible issues - ie, we know what it's not. The only way Howard, Visvanath, or myself could find out for sure, is to have access. Howard works for a living, so does Visvanath (I think). Me ... am retired but doesn't mean am not busy earning a 'retired living'.
Still think installing a monitor ON the same server having such issues won't help.
You keep saying 'it fixed itself' ... well, sorry, beg to differ, think it whatever processes/access that caused it stopped. Haven't seen a server yet that was truly 'self healing'.
If you think it's a another computer(s)/botnet involved in a denial of service attempt, the the only way I know to even get a clue about that is to watch the access.log of the apache server in realtime ... at the time these events occur ... hard to do.
Howard asked how far down did those apache references go .... not sure top can do that, but this will:
ps aux |grep apache2
That will display as text that you can copy to clip board and paste to a text file.
Sorry to ask stupid questions, but is this server in a VMWare environment? or other virtualized environment? in other words, a guest OS/Server on a server that runs other guest servers.
The access and error logs on a typical Ubuntu box are:
They normally have date/time stamps on the log entries something like:
x.x.x.x - - [24/Nov/2015:15:42:14 -0600]
If one did this:
fgrep "[24/Nov/2015:15:" /var/log/apache2/access.log > 24Nov20153PM.txt
using the date and hour the server was slow, it would create a text file of all references for that date ... the time is 24 hour clock in example.
That could result in a large text file which one could then open with nano (an editor):
nano 24Nov20153PM.txt and then be able to get an idea of there was way too much traffic from a single IP address or what was being hit.
Also, other processes that might be 'hungry' ... is there a cron job scheduled for prime time ... like a backup process/job. Backup software has been known to cause similar issues.
Do you have autobackups running in Moodle?
The other tool I've used is iptraf ... realtime network traffic.
sudo apt-get install iptraf [ENTER]
To run: iptraf [ENTER]
Choose first option in menu (monitor all interfaces). That will/might scroll so fast you can't see IP's, the TCP/IP port, etc. so under configure one could turn on logging and iptraf will capture a text file each time you use it. You let it run a while, then exit. Then cat the log file.
Sorry ... like I said ... best guesses in a hands off remotely sudo Vulcan Mind Meld via forum postings. :\
'spirit of sharing', Ken