I am seeing spikes in traffic five or six times the normal level.
We have several thousand users so I do not think that, for example, a single class of students logging on at the same time would be the cause of the problem.
In the worst cases Moodle go's down with the error:
There is a problem with moodle getting error message.
"Error: Database connection failed. It is possible that the database is overloaded or otherwise not running properly. The site administrator should also check that the database details have been correctly specified in config.php"
Spikes last a few minutes before service resumes.
I am using version 1.8.1 and am wondering if there are any known issues that could cause this, for example are there any pages that can go into a loop sending high numbers of requests back to the server?
Try adding members to a group within a large course and see what that does. There's a thread in the groups forum about how it can hang for 2-3 mins.
Check your Mysql config files. Tweak the values for total connections allowed, and or the connection time out values. Also check to see what the total number of open files is on the server. The database may be constrained by this value. Make this value very large 200,000 or more if possible.
Are you running stats? This can cause spikes. And if the stats are not finishing correctly they can start running at times other than the scheduled time. Even though your spikes are only lasting a few minutes, I would consider this as a possible cause. Look in the Moodle docs for stats and: