I need to set up moodle (1.9.4) for this scenario:
- 20.000 registered users
- 1.000 concurrent users
- All courses in format SCORM, using flash resourcing (SWF)
My questions are:
1) Operation System: It is appropriate to use the Windows Server 2003 or is essential to use Linux?
2) Database: Can MySQL support it?
3) Concurrent users: "The general rule of thumb is that Moodle can support 50 concurrent users for every 1GB of RAM" (http://docs.moodle.org/en/Installing_Moodle#Hardware). Can the number of concurrent users may increase if database is on another server?
4) Hardware: Can this settings support my moodle instalation?
- web server: Intel Xeon 2.40GHz com 8Gb RAM
- database server: Intel Xeon 2.40GHz com 4Gb RAM
Other details: I'll use Apache 2.2 / PHP 5.2 / MySQL 5.0
Any help appreciated. Thanks.
Re: News instalation (20k registered users / 1k concurrent users)
Database: Can MySQL support 20k users? Yes. I have 30,000 user accounts on my MySQL server (2x quad-core Xeon, 16GB RAM). It is handling the load with absolutely no trouble at all, although our concurrency is not as high as your requirements specify.
Concurrent users: I think the rule of thumb assumes "for every 1GB of RAM *dedicated to Apache*". For a system that large, I wouldn't even consider running the database on the same server.
Hardware: How many cores on those Xeons? An opcode cache (APC, eAccelerator, etc.) is essential, so make sure to plan for its memory use as well. 8GB of RAM seems a bit small to me. RAM is cheap, there's no reason not to load the server up with as much RAM as you can. You may also want to look at an alternate/lightweight alternative to Apache (like nginx or lighttpd) that has a lower memory footprint and can fee up more RAM for cache and additional user sessions.
Re: News instalation (20k registered users / 1k concurrent users)
However I would suggest running a newer version of Moodle, there have been a number of important improvements and security fixes recently (in the last week!), and moving up to the current version from your existing version shouldn't be too much of a headache.
Re: News instalation (20k registered users / 1k concurrent users)
> I need to set up moodle (1.9.4) for this scenario:
> - 20.000 registered users
That number alone doesn't say much.
> - 1.000 concurrent users
Yes, that is on the high side, depending on how you define "concurrent users" http://docs.moodle.org/en/Performance_FAQ#How_do_you_define_.22concurrent_users.22.3F
> - All courses in format SCORM, using flash resourcing (SWF)
That has more of an impact on the bandwidtch (and the client CPU 'course). If all your 1 k users are really concurrent, the theoritical maximum per user through a 1 Gbit/s network interface is only 1 Mbit/s!
> 1) Operation System: It is appropriate to use the Windows Server 2003 or is essential to use Linux?
As you must have noticed, Linux is considered more efficient by most of the contributors of this forum, see http://docs.moodle.org/en/Performance#Operating_System . People argue that if you master Windows, you can achieve the same. Well, if you do, you won't be asking, right?
BTW, it is operating system. Operations are more serious things
> 3) Concurrent users: "The general rule of thumb is that Moodle can support 50 concurrent users for every 1GB of RAM" (http://docs.moodle.org/en/Installing_Moodle#Hardware).
Means you need 20 GB RAM?
> Can the number of concurrent users may increase if database is on another server?
Generally yes, because they share the work. But it won't double. For one, they won't share the load equally, secondly, network latency comes in to the picture. Still, this is a simple solution (apart from more RAM) compared to real multiserver techniques.
Re: News instalation (20k registered users / 1k concurrent users)
Greg and James:
Operation System: I'm thinking about using Linux (Ubuntu).
Database: I'm going to use MySQL, thanks.
About concurrent users: How many concurrent users can I support having 16gb RAM at the webserver and 16gb RAM at the database server? (I'm guessing 800)
Hardware: My processor is a 2x Intel Xeon 5405 2.0 GHz Quad Core
Visvanath:
Your answer was really interesting. All my 1k users are really concurrent (so it means I need 20gb RAM, right?).
As I said, I'm looking for a Linux version (maybe Ubuntu).
***************************
My hardware set up
Web server
2x Intel Xeon 5405 2.0 GHz Quad Core
250 GB IDE/SATA HDD
16 GB RAM
Database server
2x Intel Xeon 5405 2.0 GHz Quad Core
250 GB IDE/SATA HDD
16 GB RAM
****************************
Thank you.
Renato
Re: News instalation (20k registered users / 1k concurrent users)
Webservers config (x3):
250 GB IDE/SATA HDD
Second Intel Xeon 5405 2.0 GHz Quad Core Processor
8 GB RAM
Windows Server 2003 - Enterprise - 64 Bit
Database config:
Second Intel Xeon 5405 2.0 GHz Quad Core Processor
146 GB 15K RPM SCSI/SAS HDD
16 GB RAM
Windows Server 2003 - Enterprise - 64 Bit
Thanks again.
Re: News instalation (20k registered users / 1k concurrent users)
You are welcome!
Two things, before there is a misunderstanding:
> All my 1k users are really concurrent
There is another factor: One has to know what there 1 k users are doing. Requesting a normal course page is one thing, submitting an online-test (Quiz module) is something different. Or, those 1 k chatting through the Chat-module would be a headache.
> (so it means I need 20gb RAM, right?).
That questionmark is mine ;) meaning I don't know the answer.
Re: News instalation (20k registered users / 1k concurrent users)
Just thought I would throw that idea out into this forum. Do people think that is a useful number?
Re: News instalation (20k registered users / 1k concurrent users)
> Number of rows written to the mdl_log table per minute
Bit too Moodle-specific for me. And, why per _minute_?
I'm thinking in the direction of "number of http-requests pending at a given instant" for a pint measure or "number of http-requests server during a minute (or second or hour)" if one needs an average.
HI all,
Sometimes a simple request hides a deceptively complex issue.
Road engineers often "count cars", for traffic managment reasons, but this does not give the information on the impact on the road itself.
The weight or load of the vehicles has much to do with the demand placed on the road surface.
In a similar way, the payload of any http request / response may make a significant difference on the demand placed on the server.
For instance, a simple file request is quite different to a Moodle Report request. The file request places a minimum of demand for authentication etc, and rest is handled quite quickly using low level features of the OS. A small file ( eg text outline ) compared to a 10Mb video will also make a difference of course.
The Report however may require a significant amount of resource as the DB searches and retrieves data, and then formatting it.
So the nature of the request may make a HUGE difference on your server operation. Simply "counting cars" is not a good measure.
In my experience, you can anticipate an overall scale, but use it to define the architecture of your server cluster, as already outlined in earlier posts. You must however be prepared to ramp up your resources ( additional web servers, larger DB server etc ) should the demand experienced be found to be higher than initially expected.
Likewise it could be less. <g>
With Moodle it is a case of the users ( and that includes the course creators and teachers ) ultimately determining the demand on your server, and that can change easily overnight. So be prepared. ( or as they say in the movies, "be afraid, very afraid!!" <g>)
Enjoy.
Gary
If we follow the road engineers, what the customer would like to know is how many small cars, how many limousines and how many trucks can pass during a given time intervall.
It would be nice if you could translate that into three Moodle (unit) loads, like page-request, file-request and quiz-request and describe how to measure them seperately. There a few hints in this script http://moodle.org/mod/forum/discuss.php?d=57028
Re: News instalation (20k registered users / 1k concurrent users)
Hi, every one I have the next issue. We have installed Moodle 1.9.5+ I have 6000 users and approximately 1000 concurrent users. In one server I have the data base Mysql and in the other one my web application Appache.
And when I have more than 500 concurrent user with more than 300 hundred requests, the Web Application, the server goes down.
I share we what we have installed and all the tunnigs we have done, Any advice or help will be appreciated.
The server A with the Web Application is:
- Procesador Intel Quad Core 3 ghz x 2 ( two procesors Quad Core )
- At the begening Ram 4 GB , now 8 GB
- Hard Disk 114 gb SATA
- Software: Fedora 12, Apache 2.2.13, PHP 5.3.1
The Server B with the Data Base is:
- Intel Quad Core 3 ghz
- Ram 8 GB
- HD 114 gb SATA
- Software: Fedora 12, Apache 2.2.13, MySQL 5.1.40
250 Users consum 1.2 GB Ram from de Application Server I had free 3 GB. But even that the application fell Here is a brief summary with stadistics which describe what it's happening.
This was the result when I had 300 concurrent users in my production environment.
Here I don't have problem
Apache Server Status for 10.36.193.7
Server Version: Apache/2.2.13 (Unix) DAV/2 PHP/5.3.1
Server Built: Oct 27 2009 15:15:20
Current Time: Saturday, 20-Feb-2010 13:01:56 CST
Restart Time: Saturday, 20-Feb-2010 13:00:28 CST
Parent Server Generation: 0
Server uptime: 1 minute 27 seconds
Total accesses: 1172 - Total Traffic: 24.3 MB
CPU Usage: u14.32 s1.73 cu0 cs0 - 18.4% CPU load
13.5 requests/sec - 286.0 kB/second - 21.2 kB/request
53 requests currently being processed, 61 idle workers
BUT THEN we simulate over the 300 concurrent users we had on that moment other 300 hundred concurrent users, givig 2 click p/s.
And the result is that thet the CPU load never reach 90% or 100% just fell down sudenñy at the 50%
Project and Scenario Comments, Operator Test Setup Test Type: TIME (run test for 1 minutes) User Simulation: 300 simultaneous users - 2 seconds between clicks Logging Period:Log every 5 seconds
1Apache Server Status for 10.36.193.7
Server Version: Apache/2.2.13 (Unix) DAV/2 PHP/5.3.1
Server Built: Oct 27 2009 15:15:20
Current Time: Saturday, 20-Feb-2010 12:42:22 CST
Restart Time: Saturday, 20-Feb-2010 12:41:40 CST
Parent Server Generation: 0
Server uptime: 42 seconds
Total accesses: 1215 - Total Traffic: 15.8 MB
CPU Usage: u15.8 s1.96 cu0 cs0 - 42.3% CPU load == Here is normal
28.9 requests/sec - 384.3 kB/second - 13.3 kB/request
347 requests currently being processed, 66 idle workers === normal
Apache Server Status for 10.36.193.7
Server Version: Apache/2.2.13 (Unix) DAV/2 PHP/5.3.1
Server Built: Oct 27 2009 15:15:20
Current Time: Saturday, 20-Feb-2010 12:42:37 CST
Restart Time: Saturday, 20-Feb-2010 12:41:40 CST
Parent Server Generation: 0
Server uptime: 57 seconds
Total accesses: 1217 - Total Traffic: 15.9 MB
CPU Usage: u15.8 s1.96 cu0 cs0 - 31.2% CPU load === But here fell down
21.4 requests/sec - 284.8 kB/second - 13.3 kB/request
376 requests currently being processed, 47 idle workers == Mean while The number of requests increses
Apache Server Status for 10.36.193.7
Server Version: Apache/2.2.13 (Unix) DAV/2 PHP/5.3.1
Server Built: Oct 27 2009 15:15:20
Current Time: Saturday, 20-Feb-2010 12:42:48 CST
Restart Time: Saturday, 20-Feb-2010 12:41:40 CST
Parent Server Generation: 0
Server uptime: 1 minute 8 seconds
Total accesses: 1220 - Total Traffic: 16.2 MB
CPU Usage: u15.8 s1.96 cu0 cs0 - 26.1% CPU load
17.9 requests/sec - 243.3 kB/second - 13.6 kB/request
391 requests currently being processed, 51 idle workers
Apache Server Status for 10.36.193.7
Server Version: Apache/2.2.13 (Unix) DAV/2 PHP/5.3.1
Server Built: Oct 27 2009 15:15:20
Current Time: Saturday, 20-Feb-2010 12:43:00 CST
Restart Time: Saturday, 20-Feb-2010 12:41:40 CST
Parent Server Generation: 0
Server uptime: 1 minute 20 seconds
Total accesses: 1222 - Total Traffic: 16.4 MB
CPU Usage: u15.8 s1.96 cu0 cs0 - 22.2% CPU load
15.3 requests/sec - 209.5 kB/second - 13.7 kB/request
417 requests currently being processed, 47 idle workers
Apache Server Status for 10.36.193.7
Server Version: Apache/2.2.13 (Unix) DAV/2 PHP/5.3.1
Server Built: Oct 27 2009 15:15:20
Current Time: Saturday, 20-Feb-2010 12:43:16 CST
Restart Time: Saturday, 20-Feb-2010 12:41:40 CST
Parent Server Generation: 0
Server uptime: 1 minute 36 seconds
Total accesses: 1237 - Total Traffic: 16.8 MB
CPU Usage: u15.8 s1.96 cu0 cs0 - 18.5% CPU load
12.9 requests/sec - 179.5 kB/second - 13.9 kB/request
456 requests currently being processed, 51 idle workers
Apache Server Status for 10.36.193.7
Server Version: Apache/2.2.13 (Unix) DAV/2 PHP/5.3.1
Server Built: Oct 27 2009 15:15:20
Current Time: Saturday, 20-Feb-2010 12:43:32 CST
Restart Time: Saturday, 20-Feb-2010 12:41:40 CST
Parent Server Generation: 0
Server uptime: 1 minute 52 seconds
Total accesses: 1238 - Total Traffic: 16.9 MB
CPU Usage: u15.8 s1.96 cu0 cs0 - 15.9% CPU load
11.1 requests/sec - 155.0 kB/second - 14.0 kB/request
491 requests currently being processed, 57 idle workers
Apache Server Status for 10.36.193.7
Server Version: Apache/2.2.13 (Unix) DAV/2 PHP/5.3.1
Server Built: Oct 27 2009 15:15:20
Current Time: Saturday, 20-Feb-2010 12:43:45 CST
Restart Time: Saturday, 20-Feb-2010 12:41:40 CST
Parent Server Generation: 0
Server uptime: 2 minutes 5 seconds
Total accesses: 1239 - Total Traffic: 17.1 MB
CPU Usage: u15.8 s1.96 cu0 cs0 - 14.2% CPU load
9.91 requests/sec - 139.9 kB/second - 14.1 kB/request
538 requests currently being processed, 47 idle workers
Apache Server Status for 10.36.193.7
Server Version: Apache/2.2.13 (Unix) DAV/2 PHP/5.3.1
Server Built: Oct 27 2009 15:15:20
Current Time: Saturday, 20-Feb-2010 12:44:05 CST
Restart Time: Saturday, 20-Feb-2010 12:41:40 CST
Parent Server Generation: 0
Server uptime: 2 minutes 25 seconds
Total accesses: 1241 - Total Traffic: 17.4 MB
CPU Usage: u15.8 s1.96 cu0 cs0 - 12.2% CPU load
8.56 requests/sec - 122.7 kB/second - 14.3 kB/request
583 requests currently being processed, 48 idle workers
FOLLOWING THE NEXT RULE
MaxClients = Total available memory * 80% / Max memory usage of apache
WE APPLIED THE TUNNINGS OVER APACHE
FEB 11
- Timeout 300
- MaxKeepAliveRequests 100
- KeepAliveTimeout 15
<IfModule prefork.c>
StartServers 12
MinSpareServers 12
MaxSpareServers 30
ServerLimit 500
MaxClients 500
MaxRequestsPerChild 10
</IfModule>
<IfModule worker.c>
StartServers 12
MaxClients 500
MinSpareThreads 25
MaxSpareThreads 75
ThreadsPerChild 25
MaxRequestsPerChild 10
</IfModule>
FEB 12
- Timeout 120
- MaxKeepAliveRequests 800
- KeepAliveTimeout 5
<IfModule prefork.c>
StartServers 8
MinSpareServers 5
MaxSpareServers 20
ServerLimit 256
MaxClients 256
MaxRequestsPerChild 10
</IfModule>
<IfModule worker.c>
StartServers 2
MaxClients 150
MinSpareThreads 25
MaxSpareThreads 75
ThreadsPerChild 25
MaxRequestsPerChild 0
</IfModule>
FEB 15
- Timeout 120
- MaxKeepAliveRequests 500
- KeepAliveTimeout 5
<IfModule prefork.c>
StartServers 45
MinSpareServers 45
MaxSpareServers 30
ServerLimit 1000
MaxClients 1000
MaxRequestsPerChild 10
</IfModule>
<IfModule worker.c>
StartServers 12
MaxClients 500
MinSpareThreads 25
MaxSpareThreads 75
ThreadsPerChild 25
MaxRequestsPerChild 10
</IfModule>
FEB 18
- Timeout 120
- MaxKeepAliveRequests 500
- KeepAliveTimeout 5
<IfModule prefork.c>
StartServers 45
MinSpareServers 45
MaxSpareServers 60
ServerLimit 1000
MaxClients 1000
MaxRequestsPerChild 10
</IfModule>
<IfModule worker.c>
StartServers 45
MaxClients 1000
MinSpareThreads 25
MaxSpareThreads 75
ThreadsPerChild 25
MaxRequestsPerChild 10
</IfModule>
FEB 20
- Timeout 60
- MaxKeepAliveRequests 100
- KeepAliveTimeout 2
<IfModule prefork.c>
StartServers 3
MinSpareServers 3
MaxSpareServers 5
ServerLimit 1000
MaxClients 1000
MaxRequestsPerChild 1000
</IfModule>
<IfModule worker.c>
StartServers 3
MaxClients 2400
MinSpareThreads 25
MaxSpareThreads 75
ThreadsPerChild 25
MaxRequestsPerChild 1000
</IfModule>
FEB 22
- Timeout 60
- MaxKeepAliveRequests 500
- KeepAliveTimeout 15
<IfModule prefork.c>
StartServers 2
MinSpareServers 2
MaxSpareServers 5
ServerLimit 500
MaxClients 500
MaxRequestsPerChild 0
</IfModule>
<IfModule worker.c>
StartServers 2
MaxClients 2400
MinSpareThreads 25
MaxSpareThreads 75
ThreadsPerChild 25
MaxRequestsPerChild 0
</IfModule>
The last set up was
MaxClients 2400
MaxRequestperChild 30
Keep a live time out is between 2 and 5
Lighttpd performance Always the memory keeps consuming 1 GB of the 4 GB in the average, now I have 8 GB and still fell down the application server.
In MySQL the next value maximum number of connections was set in 200.
(I don't have data base errors) and the CPU load on the data base is stablish under the 20%-30%.
The log life time setted is 7 days
In PHP memory was setted for PHP 5.2.1 in 128 MB.
And we have eAccelerator v v0.9.6-svn358-dev
Any way the problem stills happening. We don't know what else we need to do, if I need to have increase on the Server Application more memory up to 16 GB or what to do.
PLEASE HELP!! or advise.
Thanks...!!!
Re: News instalation (20k registered users / 1k concurrent users)
Hi Pablo,
could you give also settings of the MySQL engine ? usually the default tuning of MySQL is quite low for high load handling. Maybe the crash comes from blocking calls to the database API that comes unanswered due to logical limitations in settings...
We ran a sucessfull test for 1750 simultaneous users on a 3 x 1 x QuadCore culstered plant and we had to retune Mysql to accept 5000 input connections (in our case, Moodle virtualisation solution we made require an extra connection per page).
It may only be an hypothesis....
Hope it can help.
cheers.