A JMeter test plan for a quiz

A JMeter test plan for a quiz

by Tomoya Saito -
Number of replies: 10
Picture of Plugin developers
Hello.

I created an original test plan file (jmx file) in order to simulate concurrent quiz by many students.
We publish test plan file, Moodle course file, and user list (CSV ile) on the GitHub repository.

https://github.com/YU-MITC/jmeter-quiz-testplan

This test plan is based on HTTP requests during quiz in Moodle 3.7 (PHP 7.2).
Then, we use this test plan in Moodle 3.9 (PHP 7.3).
For these load tests, we use the JMeter 5.2.x.
Comparing the test plan generated by the Moodle plugin with our test plan, there are differences in the advanced settings of each request (about user authentication) and cookie manager.

Note that, when you use this test plan, you should execute the JMeter program on command line mode.
If you execute the JMeter on GUI mode, test plan will be overwritten.
Average of ratings: Useful (2)
In reply to Tomoya Saito

Re: A JMeter test plan for a quiz

by Visvanath Ratnaweera -
Picture of Particularly helpful Moodlers Picture of Translators
Hi Tomoya

This is a much needed addition to the tools on Moodle performance testing. Load testing a quiz is something often comes up here.

May be you add this to the documentation? Although there is (work-in-progess) documentation of the tools in Moodle core, https://docs.moodle.org/39/en/Test_course_generator and https://docs.moodle.org/39/en/JMeter_test_plan_generator specifically, there is very little in https://docs.moodle.org/dev/Load_testing_Moodle_with_JMeter. Creating one's own JMeter test plans and running them independently are completely missing there.

P.S. The construction around JMeter in Moodle Docs is halted right now. See

[Moodle Docs] Documentation dev/JMeter, XX/en/Test_course_generator and XX/en/JMeter_test_plan_generator overlapping. But that shouldn't bother documenting the part independent of the Moodle core.


In reply to Visvanath Ratnaweera

Re: A JMeter test plan for a quiz

by Shufeng Bai -
I also many people asking about running a large number of quiz concurrently. I am wondering how those people ended yet testing their setup.
It would be great if the test course generator and test plan generator can include a quiz as part of the test plan, it would benefit the community greatly.
In reply to Shufeng Bai

Re: A JMeter test plan for a quiz

by Tomoya Saito -
Picture of Plugin developers
I'm sorry for my too late response.

When I created own test plan, I don't use the "JMeter test plan generator" plugin.

I only used the JMeter.

The JMeter proxy can record HTTP queries.

At first, I went to my Moodle site via the JMeter proxy and did a series of operations for quiz.
Operations includes "login", "visit course", "visit quiz page", "start quiz", "send answer", "view review page", "move to course", and "logout".
Then, a JMX file was created.

Second, I removed unnecessary HTTP queries from test plan.
The JMeter do not render web page.
URLs for cached CSS and JavaScript files about "theme" are temporary, these URLs may not be available at a later date.
So, I removed these URLs from test plan.

Then, at each HTTP query, I replaced server name, Moodle root path, and values of parameters with variable names.
For quiz queries, attempt ID is used in parameter item names.
So, I used parameter item names include attempt ID and question number.

After that, I added a mechanism to set initial parameters, get command line parameters, and get session ID and attempt ID from the source of the web page (response content).
These tasks were complicated.

This procedure is very long, and complicated, and my english is poor.
So, it's hard for me to explain these details.

Regards,
Average of ratings: Useful (1)
In reply to Tomoya Saito

Re: A JMeter test plan for a quiz

by Shufeng Bai -
Dear Saito san,
Thank you for your great work. I manage to get it working last year and it was very useful. We were running an online physics contest last November and I had to make sure we can host enough participants before launching the contest. It helped us to identify a problem and our hosting has fixed that before the show. Thank you again for your great contribution and the instruction is crystal clear.
In reply to Shufeng Bai

ตอบ: Re: A JMeter test plan for a quiz

by Prush Sa-nga-ngam -
Hi Shufeng,

I tried this test with Moodle 3.9, but the results show many errors. Could you please advise me on how to fix them.
Attachment Screen Shot 2565-06-13 at 10.35.56.png
In reply to Prush Sa-nga-ngam

Re: ตอบ: Re: A JMeter test plan for a quiz

by Tomoya Saito -
Picture of Plugin developers
Hello,

At first, please execute your JMeter simulation at command line interface.
If you use GUI mode when you execute the simulation, the JMeter will overwrite test plan file.

Second, this test plan cannot simulate the behavior of more than 1000 students.
If you want simulate the behavior of more than 1000 students, you should modify "testuser.csv" and add more students to your Moodle site.

Next, this test plan does not support simulation iterations.
So, you cannot repeat the simulation while one execution of the JMeter command.

Moreover, many login errors occurred in your simulation.
In that case, JMeter cannot get "sesskey" value for each student.
If all students login safely, each student have his/her own sesskey value, and, URLs of numbers 73 and 74 are different for all students.
Please confirm settings of your Moodle web server (maximum number of connections, and maximum number of open files, etc.).

If you use this test plan correctly and your web server can open enough number of connection ports, and still get login errors, your web server may not be performing well.
In general, HTTP service software (apache, nginx, etc.) is not good at opening/closing a large number of connections instantaneously.
These connection errors are especially common in environments where the web server performs HTTPS encryption / decryption.
In such cases, actual web browser will retry the connections, but the JMeter client will not.
Such errors occur at network interface layer, and the service software cannot detect the error.
So that, the errors does not record in log files of service software and operating system, and administrators are less likely to notice the lack of performance of their web server.

We use Haproxy (load balancer) even under an environment of one Moodle web server, in order to avoid the errors.
And, our Haproxy performs HTTPS encryption / decryption.
Even in our environments, the same errors occur between the load balancer and the web server.
But, the Haproxy can retry connections, and our simulation will proceed normally.
Average of ratings: Useful (1)
In reply to Tomoya Saito

Re: A JMeter test plan for a quiz

by Gaurav Mahajan -
Hi Saito san,
May you share the script ?
I have had a trouble in creating one for my college thesis. It will be a great help if you share your script.
In reply to Gaurav Mahajan

Re: A JMeter test plan for a quiz

by Tomoya Saito -
Picture of Plugin developers
I do not have a script to share.
At first, administrator downloads files from the GitHub repository.
The URL of the repository is written in top comment of this topic.
Second, the administrator restores sample course into their Moodle site by using downloaded backup file.
And, administrator adds users to their Moodle site by using downloaded CSV file, and enrolls users to restored course.
Next, the administrator should modify downloaded JMX file (test plan).
Specifically, the administrator should modify parameter values defined in "User defined Variables" item.
Next, the administrator executes "jmeter" command at command line mode (e.g. terminal/console on Linux OS).
All of the above steps and an execution sample are described in the user manual (installation_manual.pdf).
No other operations or scripts are required.

By the way, I did not use any scripts etc. when creating the test plan file (JMX file).

Average of ratings: Useful (1)
In reply to Tomoya Saito

Re: A JMeter test plan for a quiz

by Gaurav Mahajan -
Moodle versions 3.9 and 4.0
I have tried making a script for concurrent users taking quiz using HTTP test recorder.
I am not providing any csv file, just increasing the number of threads with single login and password for authorization.
The test ran successfully even without parameterizing the attempt id, session key and question number.
But the total attempts doesn't change on the servers as the attempt id is same for all threads.
Attempt id and question number is not getting captured in the response body. So not able to use post processor regular expression extractor.
How to capture both attempt id and question number in response body? and use regular expression extractor?
In reply to Gaurav Mahajan

Re: A JMeter test plan for a quiz

by Tomoya Saito -
Picture of Plugin developers
I seem that you will not be able to run simultaneous quizzes without using CSV file.

When you create multple threads without using the CSV file, all theads may use the same username.
At the Moodle system, one user cannot have multiple sessions.
In fact, you can log-in to Moodle site with the same username via multiple PCs and multiple web browsers.
In such a case, each PC or browser have different session string.
However, whenever you do something with any of your PCs or browsers, Moodle system will assume that you are accessing it via your most recently used PC or browser.
Note that a single user cannot have multiple sessions at the same time within the Moodle system.

So that, a single user cannot use multiple login sessions at the same time.

As a result, many threads have failed to log-in in your test.
(Please see, labels 68 and 70 in test result image you posted earlier.)
If a thread fails to log in, the web page it received does not contain the "sesskey" value.
Threfore, the thread cannot get "sesskey" value.

Similarly, a single user cannot pretend to be multiple people and take a quiz at the same time.
If multiple threads use the same username, when one thread starts taking a quiz, other threads are forced to end.
(Please see label 68 and 70 in test result image you posted earlier.)
So that, all threads in your test failed taking the quiz.
(Please see label 116 in test result image you posted earlier.)

In conclusion, there is no successful way to proceed with the test without using the CSV file.
If you want to successfully run the load test, enroll the users included in the CSV file as  member of your Moodle course.
Then, you should use the CSV file for load testing.
(In our test plan, I have set up to use the CSV files for load testing.)

At first, I expect you to proceed with the load test according to user's manual.
Then, if you want to create your own test plan, you should look more carefully at our test plan file.
The mechanisms for getting "sesskey", "attemptid", and "qubaid" is described in the test plan file.
Average of ratings: Useful (1)