Hi all!
Quite a few people liked the 'perspectives' script I wrote ages ago (it's in another thread in this forum). The script was intended to compare different types of action in PHP, so as to inform development decisions. Which it does a sort of okay job of. But instead, people used it for an informal evaluation of their Moodle hardware.
I don't think it's very good at that.
The main problem is that evaluating performance from a single script isn't very useful with modern servers which are designed to do more than one thing at a time. In other words, supposing you have eight processor cores running at 1.5GHz, and somebody else has a processor of the exact same type but with only a single core running at 1.6GHz. The CPU result on the 'perspectives' script will be higher in the second case, but for a server workload, clearly the former processor is much better.
Similarly, if you put the database onto a separate machine from your webserver this will make the database requests in the perspectives script appear slower (due to time taken in network connection between the servers). You've just increased capacity, because the time the webserver is sitting waiting for the database server is time it can be spending servicing another request - but the perspectives script doesn't reflect that.
So! I decided to do something about it. In my own time, i.e. nobody's paying me for this. (Yes, I'm weird.) Anyway, I wrote a new load testing system that runs in your browser using a Java applet - because unlike PHP scripts, Java applets can easily do loads of things at once. This one makes up to 20 simultaneous requests to your server. Basically it's a convenient and visually interesting way to initiate a DoS attack.
I also wrote a page for it to request, which does some CPU stuff, some memory stuff, some database stuff, and some file stuff. Should cover most things that Moodle does; the proportions probably aren't right at all and the detail isn't realistic, but it probably doesn't matter too much.
Finally I wrapped all this up into an admin report. Want to try it? Because this places heavy load on your server, if you run it on a live system you should do it in scheduled maintenance time when the system is not accessible to users. Also by the way, there's a log entry for each request made by this system, so typically a couple thousand extra rows in mdl_log, hopefully that doesn't bother anyone.
To install, download the zip file Moodle load testing admin report from my GitHub downloads page. Unzip it and stick the 'loadtest' folder in the admin/report folder of your Moodle 1.9 install. That's it, then just go look at it in a browser that doesn't suck (on my Mac, Safari sucks, dunno why, Firefox is fine) on a computer that has Java 6 installed.
Here's an example of the results. I ran it on the OU developer server, it would be interesting to run it on our live servers, but I probably can't do that... As you can hopefully see the dev server scores around 7.94, or as we call it in the trade 'basically 8', requests per second. Woohoo.
| Requests/s (attempted) | Requests/s (actual) | Median time | Successful |
|---|---|---|---|
| 1 | 1.04 | 249 ms | 100% |
| 3 | 2.92 | 274 ms | 100% |
| 5 | 4.99 | 244 ms | 100% |
| 7 | 6.96 | 272 ms | 100% |
| 9 | 8.09 | 866 ms | 100% |
| 8 | 7.94 | 319 ms | 100% |
| 9 | 8.08 | 948 ms | 98.3% |
| 8.5 | 7.76 | 1282 ms | 100% |
| 8.25 | 7.55 | 554 ms | 100% |
As noted, this doesn't translate directly into real usage, but if somebody really does want to turn this report's number into a 'this server can handle x simultaneous users' lie statistic, it's probably a much better number to start with when constructing your lie statistic than the perspective script's one.
Note the 'probably' though. I haven't been able to run it on the live system, like I say. If the live system (4 front-end servers, separate db server) doesn't get a better score than the dev server, then this script sucks too. Dunno if I'll get to try it.
Post here if you find problems with this system, or if you like it, or you just want to compare scores. Comparisons between test systems and known-faster live ones might be interesting too...
All the source is right there on that GitHub site so you can fork it or send me patches or stuff if you feel like. It should be easy to convert the report to Moodle 2 as it really doesn't do much; if anyone wants to do so, that would be cool too.
--sam
