Many login failed when using redis

Many login failed when using redis

by Khaled Atteya -
Number of replies: 3

Hi,

I have 10 App servers , 3 DB Mariadb Galera servers, 3 Gluster servers and one Redis cache.

I launched stress tests with 2000 users for exam , but about 50% of them fail in login , and apache return 404 for /login/index.php , this is one of the log record in apache log:

10.1.155.180 54.77.24.81  - - [06/Jan/2025:13:24:08 +0300] "POST /login/index.php HTTP/1.1" 404 99361 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36" 

Note:

1- When I reduce the App servers from 10 to one server , the failed percentage reduce to about 20%

2- When I disable the redis cache for (application cache and session cache) the faild login reduced to 4 % 

What is the problem ?

Regards 

Average of ratings: -
In reply to Khaled Atteya

Re: Many login failed when using redis

by Eduardo Kraus -
Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers

Hi Khaled! How are you?

The first thing that stands out in your scenario is the 404 error, which indicates that the requested file is not being found. The issue that needs to be investigated is how the Moodle data is being accessed. Are they local on each server, or are you accessing them remotely via the network? This information is crucial because, in a cluster setup where servers are distributed, if the Moodle files (PHP and images) are not being shared among all of them, this can cause a significant impact on performance.

When you have multiple servers in a cluster and the access to the files is concentrated in one location (for example, a single server or a shared network area), Moodle suffers from slowness. The processor needs to wait for the network's response to continue processing requests, which results in overload and decreased performance, especially when there are many concurrent users.

So, where are the Moodle installation files located?

And for MoodleData, there is the Alternative File System plugin that I built to move MoodleData to external servers.

Another important point is the images, CSS, and JS of Moodle. Since they are managed by PHP, each request makes the server execute processes to generate and deliver these files, which overloads the infrastructure. I have a guide on how to improve Moodle's performance using caching, which could be helpful for your case. Take a look at Comprehensive Guide to Configuring BunnyCDN in Moodle. This can reduce the load on the server and help maintain performance even with many concurrent users.

Eduardo Kraus
Teacher and programmer

In reply to Khaled Atteya

Re: Many login failed when using redis

by Visvanath Ratnaweera -
Picture of Particularly helpful Moodlers Picture of Translators
Hi

You have had issues with your architecture in the past. I mean Reading/Writing file performance with Gluster !! and write performance issue with OCI FSS. Have they been resolved? Otherwise the present issue could be a continuation of those.
 
Either way, this sounds like a case for the Hardware and performance forum.
In reply to Visvanath Ratnaweera

Re: Many login failed when using redis

by Khaled Atteya -
This issue disappear when I disable the redis cache , so I think the gluster & DB are out of this issue , is it right ?

Thanks