I'll do a separate post for the bit I want to talk about...
1) System dashboard - tracks information about requests on certain common user pages (e.g. course page; forum view/discussion page) within the last 24 hours. This is an admin report plus additional hack on each page that we're tracking. The dashboard tracks the number of requests and it also tracks the time each one take so we can show a percentage 'over threshold' (3 seconds usually). For those requests, it also tracks how many result in error (moodle print_error function) so that we can look see if people are getting lots of errors. [You can click on the error number to see the URLs that gave errors.]
This stores information in a database table for each counted request, but there's an intermediate cache that temporarily stores it in local disk first (= very fast) then saves the information to the database every 30 seconds in a batch (= also fast), shaving several milliseconds off each request. We also use the same cache system for writes to Moodle log table, again saving several more milliseconds.
Example screenshot (this link may be temporary)
note: the ~ numbers are wrong, this is just a bug... the graphs are correct but that > threshold one should clearly be drawn as a different colour on the counted request graph... anyhow... you get the picture.
2) Performance logs - for every request, information is added to a logfile stored on local disk that contains the path of the request, the time it took, the number of database queries - basically this is the same information that is shown in the 'performance' block in the footer, if that's turned on. Writing to this logfile is really, really fast and because it's on a single server the file can't get corrupted... Then, overnight, work in the cron job combines logfiles from multiple servers into a single logfile, and finally analyses it to make a summary spreadsheet for the day. Using this page, we can see how long it takes on average to view e.g. mod/forum/view.php, how many requests took more than 3 seconds, etc.
Example screenshot (this link may be temporary)
As you can see these two features sort of overlap (part of the difference is related to logistics of operating a multi-server system) so it might not be desirable to implement them exactly as we have (well I think performance logs are fine the way we did them, but dashboard maybe not so much). But some similar monitoring can be useful/critical for running a system and also for identifying problem areas. Particularly after a software update, you can look at the 'before' and 'after' performance summaries and see if anything got worse.
Just to mention the inbuilt performance logging, activated by adding the following in your config:
define('MDL_PERF' , true); define('MDL_PERFDB' , true); define('MDL_PERFTOLOG' , true);
And the script in CVS to parse these logs:
I've attached a sample from one of our frontend servers (which has a few bits of 'private' info removed). But here is an example bit of useful output which demonstrates the pages which we need to get to scale due to the frequency of hits:
Top 10 dbqueries hogs weighted
/ hits: 70772 avg: 37.09 max: 567.00 /course/view.php hits: 38357 avg: 37.14 max: 645.00 /mod/chat/gui_header_js/jsupdate.php hits: 57696 avg: 9.13 max: 72.00 /course/category.php hits: 25592 avg: 18.07 max: 140.00 /mod/quiz/attempt.php hits: 3631 avg: 113.25 max: 601.00 /mod/assignment/view.php hits: 10846 avg: 29.89 max: 51.00 /login/environment.php hits: 103276 avg: 3.01 max: 5.00 /mod/hotpot/attempt.php hits: 1291 avg: 240.82 max: 829.00 /mod/resource/view.php hits: 16624 avg: 17.02 max: 43.00
Obviously our feature works the same way using the MDL_PERF, MDL_PERFDB stuff. Well, so maybe what's already there is good enough for most people! Also, it might be good enough for us if we ditch the idea of combining logs from the different front-end servers, & automatically generating summaries. Maybe something we don't need to port to Moodle 2...
Thanks for the info.
How can a mod expires instruction set like this be customised to work with 2.0?
empty cache: 347.7K
primed cache: 192.0K
empty cache: 611.3K
primed cache: 355.9K
205.5K of the 2.0 stuff is CSS that's not cached.
I haven't tested this so am not certain it works, but that's what it is supposed to do. Should be much faster than before and no need to configure it (for that anyhow).
With debugging to minimum, this is what I get for a clean 2.0 /my/ page:
I'm sure that the CSS and JS is supposed to be cacheable, efficiently, indefinitely and I know this was working because I recall people having trouble with theme updates where they forgot to hit the button to say they changed it... So if it's not working that is a straightforward bug.