I am heading a project for a fairly wide implementation of moodle. We're expecting no more than 25-50 concurrent connections at any moment. We're definitely going to be running Linux, Apache, PHP, and MySQL on the box. We've already made the decision to run a 2 120GB drives in RAID-1 (probably SATA drives, but who knows?). We have more bandwidth than you can shake a stick at, so the real limit here is going be the server's horsepower. We don't want to "under budget" for server horsepower, and we don't want to over budget, but we want a little room to grow and still have page load times reasonable. What would you recommend as far as "minimum/optimal" specs and "recommended" specs go.
I figured we'd need at least....
Decent Dual Processor machine (Xeons? probably in the 1.5ghz/1.7ghz range)
2GB Ram (Do we need more than that? 4GB?)
2 - 120GB hds in RAID-1 (Do you recommend IDE or SATA?)
More RAM, I'd go for 4GB.
Dual procs are nice, if you have the budget I'd go for a quad.
Or a nice option are separate database and Apache/PHP server, say two dual proc. 1U servers. This keeps a nice separation of db load from apache load, and you can configure each to take over if the other fails.
For 25-50 concurrent connections your machine is not "under budgeted", unless you are talking about a 25-50 person chat. The machine should be able to handle a Test (Quiz) of that size.
As already suggested, go for 4 GB RAM.
If the network is not the bottleneck I wonder what the ultimate limiting factor goint to be a) Processor (Apache, PHP, part of MySQL) or b) HD i/o (delivering data from the database)?
What is the raw data transfer rate of your HD subsystem, my gues is anything between 40 to 80 MByte/sec. That will define a ceiling even if you have a quad.
But if your performance is much lower than that, you should get a) a PHP-accelerator and/or b) put the database on its own machine.
As already suggested, go for 4 GB RAM.
If the network is not the bottleneck I wonder what the ultimate limiting factor goint to be a) Processor (Apache, PHP, part of MySQL) or b) HD i/o (delivering data from the database)?
What is the raw data transfer rate of your HD subsystem, my gues is anything between 40 to 80 MByte/sec. That will define a ceiling even if you have a quad.
But if your performance is much lower than that, you should get a) a PHP-accelerator and/or b) put the database on its own machine.
I am also looking at wide scale implementation of moodle, we are currently looking at potentially around 100-200 concurrent connections to the service, our current intranet box (amd duron 1600+) really isnt standing the test and something larger is needed, especially as we are planning to stream video and other resource material off the box. We are not however sure how much needs to be spent on something like this, can anyone enlighten me on "safe" specs for this kind of useage?
This is _the_ recurrent topic in this forum. Our moderators have done a nice job in summarizing all the wisdom in this wiki: http://docs.moodle.org/en/Performance
Generally I noticed three ways of looking at this problem:
1. "Best buy" approach
which is summarized by MD himself in this posting http://moodle.org/mod/forum/discuss.php?d=6920#32597
2. "By comparison" approach
ask around what people use, how much they deliver, hints about the bottlenecks etc. This is of course an ongoing discussion. So no other solution than to scan this forum. For a sample see this http://moodle.org/mod/forum/discuss.php?d=38448
3. "Benchmark" approach
This is what Joan Codina suggested in the following thread http://moodle.org/mod/forum/discuss.php?d=26739
IMHO this is the more "scientific" approach where numbers are involved against "financial view" like in a. or "balancing art" as in b. I guess there's quite an amount of work involved and as a result no formula is available yet. I hope to do some work on this line, but don't expect anything till the end of the year.
Generally I noticed three ways of looking at this problem:
1. "Best buy" approach
which is summarized by MD himself in this posting http://moodle.org/mod/forum/discuss.php?d=6920#32597
2. "By comparison" approach
ask around what people use, how much they deliver, hints about the bottlenecks etc. This is of course an ongoing discussion. So no other solution than to scan this forum. For a sample see this http://moodle.org/mod/forum/discuss.php?d=38448
3. "Benchmark" approach
This is what Joan Codina suggested in the following thread http://moodle.org/mod/forum/discuss.php?d=26739
IMHO this is the more "scientific" approach where numbers are involved against "financial view" like in a. or "balancing art" as in b. I guess there's quite an amount of work involved and as a result no formula is available yet. I hope to do some work on this line, but don't expect anything till the end of the year.
most of the box i build theses days goes like this:
tyan main board
2 X opteron (wich ever model gives the best power to cost ratio)
4 to 6 gig DDR 400 ECC REG
2 x 74gig 10k rpm sata drive (if the raid is used thoses 2 fast drive are reserved to run the OS only)
if more space is needed add
Areca SATAII raid card
SATAII RAID qualified drives (with Areca card you can go up to 24 x 500gig depending on needs budget and model of the card)
trow in you preferd linux distro and your ready to go
if space is not a probleme i like the antec titan server case because they are so quiet, if you go for rackmount then i like the chenbro case and drive cage
tyan main board
2 X opteron (wich ever model gives the best power to cost ratio)
4 to 6 gig DDR 400 ECC REG
2 x 74gig 10k rpm sata drive (if the raid is used thoses 2 fast drive are reserved to run the OS only)
if more space is needed add
Areca SATAII raid card
SATAII RAID qualified drives (with Areca card you can go up to 24 x 500gig depending on needs budget and model of the card)
trow in you preferd linux distro and your ready to go
if space is not a probleme i like the antec titan server case because they are so quiet, if you go for rackmount then i like the chenbro case and drive cage
Opteron 265 cpus are at a very good price point right now. These are dual core processors too.
I would suggest doing some server monitoring on your existing setup before commiting to anything. RAM is not the only bottle neck that will affect moodle performance. RAM primarily improves performance by caching drive performance and preventing paging, but not all Moodle sites are going to be IO bound. In fact a lot of sites are going to be CPU bound. If you haven't done so yet, you should put a php accellerator onto your existing server.
do a simple test run while the server seems slow. run top and look at your load average, cpu usage, and most importantly the breakdown of what the cpu is doing. If your cpu is maxed because it has iowait % then this is drive related and might be fixed some by more memory, but also and maybe more so by faster drives. If the cpu is maxed out more towards non-iowait then you are truly cpu bound and should look at accelleration and if that doesn't work faster processors/more processors.
Also look at how heavily your system is using swap memory, if you are using a lot of swap then this also indecates memory. Anytime you are going to swap a lot on an sql server you could have some major performance hits.
We run a system for about 3000 users base (not all are moodlers) and we share our web server with other web applications and it performs quite well for us on an dual Athlon 2200, with only 1 GB ram, but with 2 Raptor 10K SATA drives. The 10K drives definetely do improve performance for us.
I would also suggest looking at 3Ware raid cards. They have very good linux support and decent preformance for SATA based applications (they are not speed demons but a good mix or ease of use, support, and performance for the money).
Finally, as your site grows, consider a dual server setup. The things that will make your SQL performance fast are not the same things that will make your apache/php faster. For sql fast scsi 15k drives, the more in a stripe the better, for front ends lots of cpu power.
I would suggest doing some server monitoring on your existing setup before commiting to anything. RAM is not the only bottle neck that will affect moodle performance. RAM primarily improves performance by caching drive performance and preventing paging, but not all Moodle sites are going to be IO bound. In fact a lot of sites are going to be CPU bound. If you haven't done so yet, you should put a php accellerator onto your existing server.
do a simple test run while the server seems slow. run top and look at your load average, cpu usage, and most importantly the breakdown of what the cpu is doing. If your cpu is maxed because it has iowait % then this is drive related and might be fixed some by more memory, but also and maybe more so by faster drives. If the cpu is maxed out more towards non-iowait then you are truly cpu bound and should look at accelleration and if that doesn't work faster processors/more processors.
Also look at how heavily your system is using swap memory, if you are using a lot of swap then this also indecates memory. Anytime you are going to swap a lot on an sql server you could have some major performance hits.
We run a system for about 3000 users base (not all are moodlers) and we share our web server with other web applications and it performs quite well for us on an dual Athlon 2200, with only 1 GB ram, but with 2 Raptor 10K SATA drives. The 10K drives definetely do improve performance for us.
I would also suggest looking at 3Ware raid cards. They have very good linux support and decent preformance for SATA based applications (they are not speed demons but a good mix or ease of use, support, and performance for the money).
Finally, as your site grows, consider a dual server setup. The things that will make your SQL performance fast are not the same things that will make your apache/php faster. For sql fast scsi 15k drives, the more in a stripe the better, for front ends lots of cpu power.
AMD o Intel which is best? Because most of you talk about Opteron.
How well does Apache and mysql scale with multiple cpu's? how many threads do they run in parallel? one by http connection?
Do you know if hiperthreading gives a good performance result or is much better to have dual core.
How well does Apache and mysql scale with multiple cpu's? how many threads do they run in parallel? one by http connection?
Do you know if hiperthreading gives a good performance result or is much better to have dual core.
if you ask me Intel is really out of the server game cpu right now. all the server software is optimise for multiple processors. like others have said dual core opteron are the best ticket for server right now
The consensus seems to be that AMD 64bit cpus have an edge for a LAMP server (Linux Apache PHP Mysql). My research indecates that apache/php will scale very well with multiple cpus as long as enough users are hitting it so that you have at least as many requests as processors. Mysql seems to scale pretty well on 2 cpus but in the tests I have read doesn't benefit as much on a 4 way server.
Dual core should outperform hyperthreading. I have even read some benchmarks that show hyperthreading slowing down performance and have seen it suggested to turn this off in the bios. I don't remember what workloads had this issue with hyperthreading but worth looking into I suppose.
Strickly speaking you can run as many parallel processes as you have processors. But in reality you may have hundreds of processes going at any one time on a server and the OS will handle swapping the work around. Martin Langhoff has made several good posts taking about how to tune your mysql/apache connection maxes. It has to do as much with RAM limitations as anything and overall system capacity.
Dual core should outperform hyperthreading. I have even read some benchmarks that show hyperthreading slowing down performance and have seen it suggested to turn this off in the bios. I don't remember what workloads had this issue with hyperthreading but worth looking into I suppose.
Strickly speaking you can run as many parallel processes as you have processors. But in reality you may have hundreds of processes going at any one time on a server and the OS will handle swapping the work around. Martin Langhoff has made several good posts taking about how to tune your mysql/apache connection maxes. It has to do as much with RAM limitations as anything and overall system capacity.