I've been doing a lot of work recently trying to improve our moodle server's performance, and the first problem I had was to produce objective benchmarks to compare the various changes I made.
The tool I ended up using was http_load (http://www.acme.com/software/http_load/), which will repeatedly issue http requests to a web server, and measure how long the server took to respond, and whether it responded correctly.
I now know that I've managed to increase my performance by 10 to 20 times, but now I'm trying to calculate how many user's the server will support. This is difficult for me because we've never had a fully loaded server.
I wondered if somebody who knows the capacity of their server could run the same tests and post their results, or alternatively give me their URL and let me know when the server is quiet so I could run the tests.
The command I use is:
http-load -fetches 300 -parallel 10 <url-file>
With four url-files: main, theme, login & calendar. Each file has a single line, the contents of each file are:
main: http://127.0.0.1/index.php
theme: http://127.0.0.1/theme/standard/styles.php
login: http://127.0.0.1/login/index.php
calendar: http://127.0.0.1/calendar/view.php?view=month&cal_d=1&cal_m=7&cal_y=2008
The figures I have for my server, running version 1.9.1 are:
main: 25.5 requests/sec
theme: 113 requests/sec
login: 53 requests/sec
calendar: 49 requests/sec
thanks,
Anthony Wright
In reply to Anthony Wright
Re: Tool to estimate server's maximum concurrent users
by Visvanath Ratnaweera -
> I've managed to increase my performance by 10 to 20 times,
Would you share the details with us?
> http-load -fetches 300 -parallel 10
> With four url-files: main, theme, login & calendar. ...
Interesting approach. How do you solve the problem of logins?
> I'm trying to calculate how many user's the server will support. This is difficult for me because we've never had a fully loaded server.
There is another difficulty: How much ressources does a Moodle user demand? Obviously there is no such thing as _the_ Moodle user so you have to make a big assumption and that will remain the weak link of the argument.
Anyway hier is an attempt which is in use. You can see the conversion in the source: http://moodle.org/mod/forum/discuss.php?d=57028
Would you share the details with us?
> http-load -fetches 300 -parallel 10
> With four url-files: main, theme, login & calendar. ...
Interesting approach. How do you solve the problem of logins?
> I'm trying to calculate how many user's the server will support. This is difficult for me because we've never had a fully loaded server.
There is another difficulty: How much ressources does a Moodle user demand? Obviously there is no such thing as _the_ Moodle user so you have to make a big assumption and that will remain the weak link of the argument.
Anyway hier is an attempt which is in use. You can see the conversion in the source: http://moodle.org/mod/forum/discuss.php?d=57028
In reply to Visvanath Ratnaweera
Re: Tool to estimate server's maximum concurrent users
by Anthony Wright -
>> I've managed to increase my performance by 10 to 20 times,
>Would you share the details with us?
The major things I did were:
1) Use eAccelerator (speed increase x3)
2) Switch from apache to lighttpd (similar speed, but more stable under load)
3) Upgrade from 1.8 to 1.9 (speed increase x2 - x3)
4) Upgrade the CPU to a quad core with about x3 the power
One of the things which I discovered was that the database (postgres) is CPU bound. Which seems strange, but when you look at the sql it's understandable. Is this a common occurance?
>> http-load -fetches 300 -parallel 10
>> With four url-files: main, theme, login & calendar. ...
>Interesting approach. How do you solve the problem of logins?
Not sure I understand the question, but all these urls can be reached without logging in if that was the question.
>There is another difficulty: How much ressources does a Moodle user demand? Obviously there is no such thing as _the_ Moodle user so you have to make a big assumption and that will remain the weak link of the argument.
Totally agree, but thought with input from other sites it might be possible to make an approximate relationship between the http_load figures & the number of concurrent users.
>Anyway hier is an attempt which is in use. You can see the conversion in the source: >http://moodle.org/mod/forum/discuss.php?d=57028
Thanks for the link, it's an interesting read. What I did notice was that they're measuring the performance of various internal subsystems, and extrapolating that out to concurrent users. This requires not only the fudge factor you describe above for how much resource an active user requires, but also a fudge factor for how efficiently these internal subsystems are used. Measuring http requests/second measures the performance of the system at a much closer point to the user, and would seem to me to be capable of giving a more accurate result.
For example, when I measured http requests/seconds I noticed a x3 speed improvement when I introduced eAccelerator, but perspective only measures the number of function calls that php can make in a second which won't be affected much by eAccelerator. Perspective won't see any improvement from 1.8 to 1.9 because it doesn't measure the efficiency of the code, again I saw a significant improvement. Finally while things have improved with 1.9, performance has a nasty habit of degrading as the number of courses increases. Again perspective won't see this, but measuring http requests/second will.
The two big problems I have with the http requests/second approach is whether the urls I'm requesting are sensible ones to use to benchmark a server (maybe I should use more urls to benchmark different parts of the system), and what the fudge factor is for going from requests/second to number of concurrent users.
Trying to determine this fudge factor is where I could really benefit from other people's input by measuring the performance of a number of servers where the admins know how many users it will support, hence this post. All I need is the server's URL, number of concurrent users it supports & a quiet time when it would be ok to do the tests (it takes 2-3 minutes). I'll post the results back here.
thanks,
Anthony Wright.
>Would you share the details with us?
The major things I did were:
1) Use eAccelerator (speed increase x3)
2) Switch from apache to lighttpd (similar speed, but more stable under load)
3) Upgrade from 1.8 to 1.9 (speed increase x2 - x3)
4) Upgrade the CPU to a quad core with about x3 the power
One of the things which I discovered was that the database (postgres) is CPU bound. Which seems strange, but when you look at the sql it's understandable. Is this a common occurance?
>> http-load -fetches 300 -parallel 10
>> With four url-files: main, theme, login & calendar. ...
>Interesting approach. How do you solve the problem of logins?
Not sure I understand the question, but all these urls can be reached without logging in if that was the question.
>There is another difficulty: How much ressources does a Moodle user demand? Obviously there is no such thing as _the_ Moodle user so you have to make a big assumption and that will remain the weak link of the argument.
Totally agree, but thought with input from other sites it might be possible to make an approximate relationship between the http_load figures & the number of concurrent users.
>Anyway hier is an attempt which is in use. You can see the conversion in the source: >http://moodle.org/mod/forum/discuss.php?d=57028
Thanks for the link, it's an interesting read. What I did notice was that they're measuring the performance of various internal subsystems, and extrapolating that out to concurrent users. This requires not only the fudge factor you describe above for how much resource an active user requires, but also a fudge factor for how efficiently these internal subsystems are used. Measuring http requests/second measures the performance of the system at a much closer point to the user, and would seem to me to be capable of giving a more accurate result.
For example, when I measured http requests/seconds I noticed a x3 speed improvement when I introduced eAccelerator, but perspective only measures the number of function calls that php can make in a second which won't be affected much by eAccelerator. Perspective won't see any improvement from 1.8 to 1.9 because it doesn't measure the efficiency of the code, again I saw a significant improvement. Finally while things have improved with 1.9, performance has a nasty habit of degrading as the number of courses increases. Again perspective won't see this, but measuring http requests/second will.
The two big problems I have with the http requests/second approach is whether the urls I'm requesting are sensible ones to use to benchmark a server (maybe I should use more urls to benchmark different parts of the system), and what the fudge factor is for going from requests/second to number of concurrent users.
Trying to determine this fudge factor is where I could really benefit from other people's input by measuring the performance of a number of servers where the admins know how many users it will support, hence this post. All I need is the server's URL, number of concurrent users it supports & a quiet time when it would be ok to do the tests (it takes 2-3 minutes). I'll post the results back here.
thanks,
Anthony Wright.
In reply to Anthony Wright
Re: Tool to estimate server's maximum concurrent users
by Visvanath Ratnaweera -
> The major things I did were:
> 1) Use eAccelerator (speed increase x3)
> 2) Switch from apache to lighttpd (similar speed, but more stable under load)
> 3) Upgrade from 1.8 to 1.9 (speed increase x2 - x3)
> 4) Upgrade the CPU to a quad core with about x3 the power
Thanks for the list, very useful.
> One of the things which I discovered was that the database (postgres) is CPU bound. Which seems strange, but when you look at the sql it's understandable. Is this a common occurance?
I too would have though the db is i/o bound. Possilbly the db is in the cache?
>> How do you solve the problem of logins?
> Not sure I understand the question, but all these urls can be reached without logging in if that was the question.
My point is, the Moodle users work after logging in. It affects the performance because now you have a whole lot of new SQL querries. Any Moodle-specific benchmarking should take that into account.
On the comparison perspective vs. your approach (http-requests):
1. yes, perspective is not coupled to Moodle code. As you say, it measures various "general" functions of the machine
Which makes http-requests method Moodle-specific, therefore you have to mention the version.
2. perspective runs in the same machine, http-requests should come from outside (not a must)
Network performance goes in to your measurement. So have to specify the link.
3. To convert the measurement into so many users, both approaches need a conversion factor. My question is, is it something as simple as, n http-requests per user?
Secondly you want to get that factor from different admins. Isn't too much guess work and extrapolations involved in that process?
May be a whole set of information per installation might be more useful. For example:
- hardware
- software stack (OS, Webserver, db, php, Moodle)
- a _set_ of hard benchmark figures
- estimated max. users and the formula
> 1) Use eAccelerator (speed increase x3)
> 2) Switch from apache to lighttpd (similar speed, but more stable under load)
> 3) Upgrade from 1.8 to 1.9 (speed increase x2 - x3)
> 4) Upgrade the CPU to a quad core with about x3 the power
Thanks for the list, very useful.
> One of the things which I discovered was that the database (postgres) is CPU bound. Which seems strange, but when you look at the sql it's understandable. Is this a common occurance?
I too would have though the db is i/o bound. Possilbly the db is in the cache?
>> How do you solve the problem of logins?
> Not sure I understand the question, but all these urls can be reached without logging in if that was the question.
My point is, the Moodle users work after logging in. It affects the performance because now you have a whole lot of new SQL querries. Any Moodle-specific benchmarking should take that into account.
On the comparison perspective vs. your approach (http-requests):
1. yes, perspective is not coupled to Moodle code. As you say, it measures various "general" functions of the machine
Which makes http-requests method Moodle-specific, therefore you have to mention the version.
2. perspective runs in the same machine, http-requests should come from outside (not a must)
Network performance goes in to your measurement. So have to specify the link.
3. To convert the measurement into so many users, both approaches need a conversion factor. My question is, is it something as simple as, n http-requests per user?
Secondly you want to get that factor from different admins. Isn't too much guess work and extrapolations involved in that process?
May be a whole set of information per installation might be more useful. For example:
- hardware
- software stack (OS, Webserver, db, php, Moodle)
- a _set_ of hard benchmark figures
- estimated max. users and the formula