Site "speed" metrics

Site "speed" metrics

by Howard Miller -
Number of replies: 7
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

What metrics do people out there use to measure the "speed" of their Moodle systems and how do they measure it...

I'm thinking of things like average page load time. 

Where I'm coming from is that if we make changes to the system, excluding massive and obvious differences, how do we know if we have improved the speed/responsiveness of the site vs. made it worse?

Average of ratings: Useful (1)
In reply to Howard Miller

Re: Site "speed" metrics

by sam marshall -
Picture of Core developers Picture of Peer reviewers Picture of Plugin developers

Hi Howard,

We have systems that record page load time based on web server logs. It's not the perfect measure given that it doesn't count time for JavaScript to load and display etc., but it's pretty good for indicating server performance.

(These are available live, we use an internal tool and a commercial one called Splunk, which can be used to spot live problems.)

Just for example our course view page (which is a pretty complicated page on our systems as we tend to have added stuff on it) is 0.7 seconds median at the moment. Content page view is 0.5 seconds median. (95th percentile for both is around about 2.5/1.5 seconds.)

We also have Apache JMeter load testing plans where we run a high load of our most common page views on a pair of performance test systems. Prior to deploying each new OU Moodle version, we do before/after comparisons and look at the results for page load time using those same systems. Normally it's no different or slightly slower - if it's a lot slower we would worry.I think Moodle run similar performance tests each release, don't they? I couldn't find it with a quick search.

--sam

Average of ratings: Useful (3)
In reply to Howard Miller

Re: Site "speed" metrics

by Leon Stringer -
Picture of Core developers Picture of Particularly helpful Moodlers

I've always just timed a user login and this will hit the web server, Moodledata and database. I take three timings and use the average. I do one login first and discard the timing to ensure the server-side cache is initialised beforehand. Then I make what every change I'm testing and repeat the test. I realise it's pretty unsophisticated but it's quick and easy and has been useful in showing when things are getting slower or faster.

Average of ratings: Useful (2)
In reply to Howard Miller

Re: Site "speed" metrics

by Ken Task -
Picture of Particularly helpful Moodlers

Other responses might be more what you are looking for, but I've been tinkering with a thing called 'benchy' (php) - apache bench ... that uses php.

https://github.com/gomox/benchy

In the setup file for benchy, one provides a url to a php script.   That php script I've got right now outside of Moodle, accessing the database in the old way, but pointed to displaying all records in the log store.   Yes, that's massive so this is a non-prime time thang.

Sample of config (without credentials to access Moodle content):

[root@sos]# cat config.json
{
    "url": "https://site/benchy/sometest.php/",
    "cookie": null,
    "basicauth": null,
    "tests": [
             { "concurrency": 1,   "requests": 10   },
             { "concurrency": 2,   "requests": 20   },
             { "concurrency": 4,   "requests": 40   },
             { "concurrency": 8,   "requests": 80   },
             { "concurrency": 16,  "requests": 160  },
             { "concurrency": 32,  "requests": 320  },
             { "concurrency": 64,  "requests": 640  },
             { "concurrency": 128, "requests": 1280 }
    ]
}

One runs benchy one time.   Then one makes tweaks to DB/apache.   Then one runs benchy again.

The second graph drawn by benchy is right beside your first run.  Pretty easy to determine if faster or slower.

If you look at the screen shots from the github link above you'll see what I mean.  **  But, there are additional options one can choose from the 'benchy' menu on the left ... one is concurrent. **

What I'd like to be able to do with it .... find a way to pass credentials to Moodle ... ie, login a user ... so that I could point the php script to just the rendering of a quiz .... set to something 'heavy processing' ... like show all questions of a quiz that contains 100 questions.

It's not jmeter ... but then again ... one doesn't run jmeter on a production server where clean up of a run on a production server is/could be massive.    Benchy is outside of Moodle so possible to run it on a production server - within reason ... me thinks!

'spirit of sharing', Ken


Average of ratings: Useful (2)
In reply to Howard Miller

Re: Site "speed" metrics

by Rick Jerz -
Picture of Particularly helpful Moodlers Picture of Testers

I am wondering if this "perspective.php" method still means anything.  Someone created this awhile ago.  I use it to compare performance of different moodles.

To use this, I put this file in my main moodle folder and then initiate from my browsers (www.mydomain.com/moodle/perspective.php.)

I happened to just use this yesterday as I am working on getting a new VPS up and running.  My results showed that my new experimental VPS, running php 7.1, was much faster than my current production moodle.  However, these measurements are  not static, they change each time the script is run.  Below are my results:

For my new VPS...

Speed test

In one second you can do...

  • 17562000 function calls
  • 102000 16KB files read from disk (cache)
  • 24800 regular expression replaces over 1KB of text
  • 7300 16KB files written to disk (cache)
  • 5460 get_record calls on the course table
  • 3360 insert_record calls on the course table
  • 3520 update_record calls on the course table
  • 1.12 seconds to delete 3360 entries

Script took 14.92 seconds to execute.

For my current production VPS... (php 5.6)


Speed test

In one second you can do...

  • 2624000 function calls
  • 88100 16KB files read from disk (cache)
  • 19500 regular expression replaces over 1KB of text
  • 6700 16KB files written to disk (cache)
  • 5390 get_record calls on the course table
  • 4230 insert_record calls on the course table
  • 4570 update_record calls on the course table
  • 1.01 seconds to delete 4230 entries

Script took 14.82 seconds to execute.


Average of ratings: Useful (2)
In reply to Rick Jerz

Re: Site "speed" metrics

by Michael Penney -

Currently we use NewRelic, which gives a nice breakdown of Web Application (backend PHP/MySQL), Network, DOM processing, and Page Rendering.

In reply to Howard Miller

Re: Site "speed" metrics

by Howard Miller -
Picture of Core developers Picture of Documentation writers Picture of Particularly helpful Moodlers Picture of Peer reviewers Picture of Plugin developers

Thanks for the help everybody - some things to look into.

I think I also need to get it clear in my head if I am talking about running speed tests on a live site or one with no users. Clearly the 'real world' performance is what matters but it will take a lot more data to get some sort of sensible numbers. Or graphs probably. 

In reply to Howard Miller

Re: Site "speed" metrics

by Mathew Gancarz -
Picture of Core developers

Just to add my two cents, when we were tuning cache settings and db settings and all that, I used a jmeter test plan and graphs to measure outcomes of different settings. This can lead down a rabbit hole, since sometimes when you are tuning for one metric, another metric might go up, but what we tuned for was page load times and maximum # of concurrent users served, as defined by a certain # of fake accounts logging in an order and performing a series of 'representative student actions' such as browsing course, opening quiz, etc.


For ongoing we use a free service from gtmetrix.com that more simply measures public-facing page load times from an external location you can choose (so that we take into account internet latency issues that normal users are exposed to). It gives us a longitudinal measure when we are not actively testing/tweaking, so for example it lets us see that that spiffy new high-res image we added to the front page to feature the latest new course added 1 second on average to page load times.

Average of ratings: Useful (3)