I have a VPS where a moodle site is working (4GB, 4CPU,Centos). In its domain a production site and another subdomain a test site is functional. I would like to test how many concurrent users can take a quiz on my server. I was told about JMeter and related moodle scripts to do that. The documentation says JMeter test should not be conducted on a production server but only on development system. Can I call the second moodle site on same server as a development system?
Do I need completely different vps for testing?
From what you have described, you have virtual Apaches (domains), same server ... which means both production and development *are* using the same DB server (localhost?) and the same Apache ... production.somenet and development.somenet sites resolve to the same IP address.
Is that correct? IF so, then yes ... using jmeter on the development virtual apache/DB will have affects upon production.
Separate VPS ... same config ... yes, that would be better and avoid halting (not necessarily 'crashing') production server, but ...
One would first have to *know* about how much production site might be taxed and is it running optimally? Therefore, suggest before doing anything doing some research on your production site:
Has the production site ever been slow? If so under what conditions of usage? Was there ever a time when a course required students to be logged on and taking a 'final' between a narrow time frame ... like from 1:00 PM to 2:30PM Monday/Tuesday or Wed.?
Is your site usage acsynchronous ... where students/learners are not required to be logged on at the same time for activities. Or synchronous ... where teacher(s) is(are) in a F-2-F with students logged on at the same time? (+ X number of classes doing same thing?)
Install MySQLTuner and run it - remember you've 2 DB's on same box, but the config tweaks for the DB server will more than likely be a result of the site that's used the most ... your production site.
Then there is Apache ... how are you running? mod, fastcgi with php-fmp or some other? or are you running nginx and proxies?
Bottom line for me ... 4 gig boxen was farily easy to halt and the jmeter test I ran was never really true to usage.
But ... like I said ... 'for me' ... am certain others might add their 2 cents to this!
'SoS', Ken
Thank you very much Ken, you are indeed Saving our Soul very often here!
My site is running smoothly without any issue but I planned for an online quiz competition where about 600 students were expected to take the quiz...it was a timed moodle quiz and the site got crashed and put me down. So now I am thinking of segregating 600 students into separate groups to reduce the load. So trying to calculate how many can access at a same time.
I dont have another VPS just for testing this
so thought of doing something on the second site on the same server.
My php is Server API CGI/FastCGI, no nginx/proxies. I purchased a VPS from GoDaddy and installed Moodle using Installatron.
My site is running smoothly without any issue but I planned for an online quiz competition where about 600 students were expected to take the quiz...it was a timed moodle quiz and the site got crashed and put me down. So now I am thinking of segregating 600 students into separate groups to reduce the load. So trying to calculate how many can access at a same time.
I dont have another VPS just for testing this
My php is Server API CGI/FastCGI, no nginx/proxies. I purchased a VPS from GoDaddy and installed Moodle using Installatron.
'SoS' is intended to be short for 'Spirit of Sharing' ... but the other could be an interp as well ... sometimes.
A couple of things before asking more questions ...
Ooops! GoDaddy. X 2 with 'installed Moodle using Installatron' ...
That should bring 2 'regulars' into this discussion ... but ... since I started ...
So you have how many student actually registered and taking a class that will be used for testing? 'about 600' isn't very exacting. You can do better than that with an exact number. Why? An old general rule of thumb for moodle ... for every 20 concurrent connections =1 Gig memory.
So X memory for 'about 600' could/might get you close, but no cigar!
Then there is the test itself ... I take it X number of questions per page and total of X questions given how long to finish? Any audio/video with questions?
Comment about thought of ... "segregating [X number] students into separate groups' ... same day? time frame? Some catch 22's there you cannot control ... there could be some students who will login and refresh their screens until the quiz for their 'group' appears and is available!
Have seen a computer lab of just 12 6 grade students give a fairly powerful Moodle server fits by 'refreshing' multimedia that wasn't broken into pages. Those little darlings demonstrated the need for a lot more thought into how courses were designed/conducted!
Sweet spot for me has been 16 Gig memory - on an all in one box ... code + DB server - fairly large K12 school - about 11,000 students. Same day - nope. Over 2 days - yep - 1st days 'group' numbered fewer than 2nd by design. Server gets a breather overnight and a break for 'tweaking' if needed.
Let the 'sharing' game begin!
GoDaddy anyone? Installertron ... anyone else?
'SoS', Ken
Thanks again.
To be specific I have exactly 637 users (They belong to three groups medium/intermediate and advanced). There are three different quizzes with 60 questions each. Each quiz will be taken only specific number of users (230, 266,141 ). Out of 60 questions about 10 questions do contain a listening component each in the form of audio files (mp3 size less than 3.5MB). Now my server configuration is : 4CPU core, 8GB RAM, 200GB diskspace. I am thinking of subdividing each group based on location and duplicate quizzes with different timing so that only 8x20 concurrent users. I will schedule different times of a day with at least one hour break for different groups.
Your opinion on this please...
To be specific I have exactly 637 users (They belong to three groups medium/intermediate and advanced). There are three different quizzes with 60 questions each. Each quiz will be taken only specific number of users (230, 266,141 ). Out of 60 questions about 10 questions do contain a listening component each in the form of audio files (mp3 size less than 3.5MB). Now my server configuration is : 4CPU core, 8GB RAM, 200GB diskspace. I am thinking of subdividing each group based on location and duplicate quizzes with different timing so that only 8x20 concurrent users. I will schedule different times of a day with at least one hour break for different groups.
Your opinion on this please...
Ok, this gonna sound not helpful but ....
Ahhhh ... hints shared ... a college!
First posting in this thread: 4GB, 4CPU,Centos
In the more exacting posting ... exact number of users posting ... 4CPU core, 8GB RAM ... which is it?
"subdividing each group based on location"??
"specific number of users (230, 266,141 )". Thanks!
So your highest would be 266 users (ball park for calc 270) ... that's a lot smaller number than 600+ if you can keep the other groups out.
In olden times (brick and mortar) students were informed in advance that
grades would be posted (print out tacked to a bull board) day after test.
So your group 1 - 230's expectation would be to see grades right after completing?
Opinions don't make it so!
We are still taking about how in the Moodle without some server data ... perhaps the most significant is config and performance of the DB server!
Install MySQLTuner and run it please! DB server does have stats you've not seen!
'SoS', Ken
Thanks for your reply. I teach in a college but this server we are talking about is a community one (aka limited fund) and with my personal capacity upgraded to its max 8GB ram. As this quiz is a competition we suppressed all feedback/marks. Students will be communicated later through email with their scores. Therefore they have to just submit and go. So for 8GB the safest concurrent users is a goal.
Mean time I have attached the MySqlTuner report as suggested.
Thanks for your help.
Mean time I have attached the MySqlTuner report as suggested.
Thanks for your help.
Thanks for running. You have some data now and info which you need to research and set up in MySQL.
[ok']'s are good ... note:
[OK] Highest usage of available connections: 56% (86/151)
The above is one to watch for concurrent hints. Max connections 151 is default for MySQL/MariaDB and you can see the most you've had is 86.
[!!] are not good and require your attention
[!!] Maximum reached memory usage: 21.8G (284.90% of installed RAM)
[!!] Maximum possible memory usage: 38.1G (498.59% of installed RAM)
[!!] Overall possible memory usage with other process exceeded memory
All the above are **major** and not good. So how much RAM does server have?
[!!] InnoDB buffer pool / data size: 128.0M/700.2M
Actual data size (700.2M) is larger than the pool allocated ... 128.M ... which is default.
At the bottom of the report.txt, Tuner recommendations:
Variables to adjust:
*** MySQL's maximum memory usage is dangerously high ***
*** Add RAM before increasing MySQL buffer variables ***
query_cache_size (=0)
query_cache_type (=0)
query_cache_limit (> 1M, or use smaller result sets)
join_buffer_size (> 256.0K, or always use indexes with JOINs)
table_open_cache (> 2000)
performance_schema = ON enable PFS
innodb_buffer_pool_size (>= 700.2M) if possible.
innodb_log_file_size should be (=16M) if possible, so InnoDB total log files size equals to 25% of buffer pool size.
Read up on variables ... then decide if Tuner recommendation should be applied or not or adjusted up or down.
No ... don't ask what the values should be ... it's your server. You admin now and will have to admin in future. Time to put on the DB admin hat and learn a little DB admin! ;)
'SoS' ... spirit of sharing ... Ken
I have a GoDaddy VPS, it's running well, but I have never tried giving a quiz to 600 students by saying "Start the quiz right now." So I have no idea how my GoDaddy VPS would behave, and therefore cannot be of much help. Ken and others here have pointed you to some ideas and suggestions. I'll try to give just a few more comments (repeating some from others.)
1) Do not install Moodle using Installatron. Doing so gives you no clue about what this utility has done, and it increases the unknown.
2) On my VPS, yes, I do edit and try to improve the database's my.cnf file. I do my best, but I am not an expert in tuning databases. On my VPS, I run MariaDB. In your case, since you are subjecting your database to a heavy load ("all 600 students begin the quiz right now") fine-tuning the database would probably be helpful.
3) You might want to review some of your php settings to make sure they are in order for this kind of load. There are some "timeout" parameters. Have you done anything to your php.ini file?
4) There are many other settings on these (GoDaddy) VPSs, at the server level, that are performance-related. My courses are asynchronous, I don't typically see any heavy loads, so I don't worry about changing these server-side settings. I tend to use the default settings.
5) GoDaddy technical support, and many other companies, will not get into fine-tuning your server for a specific application. Some of these companies might do a little more for very specific applications, like WordPress, but not for other applications.
6) Someday I might be smart enough to install and run the MySQLTuner application, that Ken mentions. Right now, can't.
7) I do run another utility that was provided by someone else called "perspective.php." I have used this when I was fine-tuning my.cnf. You put this file into your moodle folder and run it, such as "www.mydomain.com/moodle/perspective.php." Below are the results for my VPS.
Speed test
In one second you can do...
- 20381000 function calls
- 93200 16KB files read from disk (cache)
- 35000 regular expression replaces over 1KB of text
- 4200 16KB files written to disk (cache)
- 15630 get_record calls on the course table
- 7860 insert_record calls on the course table
- 12410 update_record calls on the course table
- 1.01 seconds to delete 7860 entries
Script took 15.11 seconds to execute.
Got any quiz coming up ... not the biggy?
Might do something like this ... still in gathering data/info stage:
Connections script - more exacting ballpark?
[root@moodle bin]# cat connections
echo $(date +%Y%m%d%-H%M) >> /var/log/connections.log;
netstat -plan|grep :443|awk {'print $5'}|cut -d: -f 1|sort|uniq -c|sort -nk 1 >> /var/log/connections.log;
echo '-------------' >> /var/log/connections.log;
Realtime watching of connections.log might show something like the following: (172.25.x.x is private IP internal to K12 network)
20160419805 - 8:05 AM
2 172.25.84.22
10
-------------
20160419810 - 8:10 AM ... students should be in class so in the space of 5 minutes connections jumped from 10 to 350.
2 166.x.x.x.
2 172.25.84.22
350
-------------
20160419815 ... oops, some teachers might not want to use Moodle right away! Students logout.?
2 172.25.84.22
230
-------------
20160419820
2 172.25.84.22
298
-------------
20160419825
2 172.25.84.22
248
What's been observed this K12 Moodle - really educated guesses.
Fewer connections during class changes ... students log off, go physically to next class. Burst occur after the pass period (5 minute) ... HS campus is 1to1 ... every student has access to or owns a device (smartphone/tablet)/laptop .. campus is also 'bring your own'. Oh what fun!
Tuner run on k12 server ...
Significant stats/info for DB server on this K12 Moodle (unique):
[OK] Maximum reached memory usage: 3.3G (21.10% of installed RAM)
[OK] Maximum possible memory usage: 4.4G (28.44% of installed RAM)
[OK] Overall possible memory usage with other process is compatible with memory available
[OK] Slow queries: 0% (0/103M)
[OK] InnoDB buffer pool / data size: 3.0G/2.3G
[OK] Aborted connections: 0.00% (1/2781798)
'SoS', Ken
Staggering will definitely help. 600 quizzes requested the same minute or in 10 batches of 60, in 10 min intervals are hugely different. There were even ideas of micro-staggering: protect the quiz by a long, easy to remember password - like a full sentence. The differing typing speeds and having to retype due to typos will stretch wave of requests. Well, need to combine with supervision, don't allow the candidates to sit on the F5-key waiting for the exam to appear!
Oops, the GoDaddy regulars! One has done his duty outstations: https://moodle.org/mod/forum/discuss.php?d=397037&parent=1601338.
Moving to Hardware and performance forum...
Hi
So this is the Internet access to Moodle - as opposed to Intranet access? Ref. In-campus access thru intranet site and outside internet access. Could you give us an update here? (Unfortunately the original thread has been locked.)
Yes, the JMeter testing has been vastly simplified in Moodle core albeit not well documented. See Is moodle-jmeter-script-generator still maintained?. The bad news is there is no scheme for the quiz activity. So only the "discarded" rule of thumb Ken mentioned - "for every 20 concurrent connections =1 Gig memory" - still stands. 30 GB is probably not the range you expected.
BTW, when the quiz "crashed" with 600 exam candidates, were you monitoring the server? Which is the parameter which broke the server? You also mentioned some complication in your network in the old discussion I have linked above. Has it been resolved?
So this is the Internet access to Moodle - as opposed to Intranet access? Ref. In-campus access thru intranet site and outside internet access. Could you give us an update here? (Unfortunately the original thread has been locked.)
Yes, the JMeter testing has been vastly simplified in Moodle core albeit not well documented. See Is moodle-jmeter-script-generator still maintained?. The bad news is there is no scheme for the quiz activity. So only the "discarded" rule of thumb Ken mentioned - "for every 20 concurrent connections =1 Gig memory" - still stands. 30 GB is probably not the range you expected.
BTW, when the quiz "crashed" with 600 exam candidates, were you monitoring the server? Which is the parameter which broke the server? You also mentioned some complication in your network in the old discussion I have linked above. Has it been resolved?
Thank you visvanath
Unfortunately GoDaddy did not give me resources usage history. I will try to ask support anything can be found. Not sure Moodle will have logs to show (when I checked under reports, site errors on the day of quiz got around 895 rows of site errors with mostly user login failure.
Thanks for referring to my old question. I resolved that. This is another moodle site and present discussion is not about that site.. How I managed was, the server was locally hosted and local dns server point to this server using intranet. Whereas outside the college we have created subdomain on our website dns server to point our public IP of the server located in the college. Using firewall we set up in such a way when faculty/students type subdom.college.somenet in a browser within campus, they reach the moodle site using intranet. If they type same subdom.college.somenet from outside the campus they reach public IP where moodle site is reached using internet.
Unfortunately GoDaddy did not give me resources usage history. I will try to ask support anything can be found. Not sure Moodle will have logs to show (when I checked under reports, site errors on the day of quiz got around 895 rows of site errors with mostly user login failure.
Thanks for referring to my old question. I resolved that. This is another moodle site and present discussion is not about that site.. How I managed was, the server was locally hosted and local dns server point to this server using intranet. Whereas outside the college we have created subdomain on our website dns server to point our public IP of the server located in the college. Using firewall we set up in such a way when faculty/students type subdom.college.somenet in a browser within campus, they reach the moodle site using intranet. If they type same subdom.college.somenet from outside the campus they reach public IP where moodle site is reached using internet.
Hi
So the server in the intranet got also Internet access with https://docs.moodle.org/en/Masquerading ? Yes, that is the prescribed method. (You see, it is always good to close the thread once you found a solution.)
The server you are discussing has nothing to with that intranet server? But the network may be involved. Since this looks like a high stake examination you must be conducting it in the institution's network. What about the slow network connection you mentioned in the previous discussion? Do these 600 candidates, some downloading audio, go through that slow link?
So the server in the intranet got also Internet access with https://docs.moodle.org/en/Masquerading ? Yes, that is the prescribed method. (You see, it is always good to close the thread once you found a solution.)
The server you are discussing has nothing to with that intranet server? But the network may be involved. Since this looks like a high stake examination you must be conducting it in the institution's network. What about the slow network connection you mentioned in the previous discussion? Do these 600 candidates, some downloading audio, go through that slow link?
Thanks for your suggestion.
my both questions discuss about completely two different servers. There is no intranet version for the server discussed in this server.
my both questions discuss about completely two different servers. There is no intranet version for the server discussed in this server.
Sorry for following a wrong trace. One never knows!
The other sub-thread is pointing to something https://moodle.org/mod/forum/discuss.php?d=397116#p1601438; we can call this sub-thread closed.
The other sub-thread is pointing to something https://moodle.org/mod/forum/discuss.php?d=397116#p1601438; we can call this sub-thread closed.