We have a Moodle 3.9 site with about 18.000 login attempts per day (measured by grep "POST /login/index.php" over apache2 access.logs). And about 1.000 "Invalid Login Token" messages per day in the error.log. The user just sees the message "Invalid login, please try again". Same as when username/password is wrong.
That number looks quite large to me. I'm wondering if this abnormal and what is causing it. Maybe you can provide me with numbers from your Moodle site?
We already applied code changes in the past addressing double submission that significantly reduced the number of invalid login token messages. One of it was to apply double submission prevention to the login form (MDL-75083). And another one had to do with a changed chromium behaviour. There are still two issues open I created that I suspect have something to do with the large amount of "Invalid Login Token" messages: MDL-75101 and MDL-75343.
What do you think? Am I chasing ghosts? What are your numbers?
Could it be a network filter? What are the network devices between the requests coming from the WAN and the Moodle server? Firewalls, load balencers, proxy servers?
Is the web server a single machine (box) or a cluster.
Is the web server a single machine (box) or a cluster.
The Moodle server is a single machine, with a separate server for the database. No load balancers or reverse proxys in front. Firewalls, yes at least two and one of it has fancy intransparent "next generation" features. :P But no explicit filtering is applied.
How could a network filter (and what kind?) produce that issue?
Well, a network filter _could_, if it wants. Moodle can not do anything about it. This token is data in the request.
Why would it do such a thing? I don't know. Ask the people who run it.
Please notice that I am not saying that this is what is happening. Sure there are other possible causes. An aggressive CDN is another.
Why would it do such a thing? I don't know. Ask the people who run it.
Please notice that I am not saying that this is what is happening. Sure there are other possible causes. An aggressive CDN is another.
Is the IP address associated with the error one that is used by a user on your system?
I get a few of them ... am using Google Oauth2 and client IP address shown in error log isn't one that shows as a user IP address:
error clip ...
[Mon Nov 21 07:08:24.691787 2022] [php7:notice] [pid 2770] [client 66.249.72.16:61497] Default exception handler: Your session has most likely timed out. blah, blah, blah.
IP address shown above is a Google IP. but query of db mdl_user table for lastip doesn't show any user using that IP.
It's annoying ... but ...if you get a bunch of them from an IP address like that (not sure how one would determine 'bunch') is the IP associated with a VPN service? If not, consider that hackers/crackers will start with a couple of pokes/probes and then escalate.
I do proactive things where I can ...
If that IP has been 'pesky':
firewall-cmd --zone=drop --add-source=$1
That's at the network layer. 
and not permanent.
'SoS', Ken
Since we have no reverse proxy in between we see the actual user IPs in the apache2 access logs of the Moodle server. And I'm not concerned about brute force attacks on user logins. I'm taking about login token errors. Not failed logins caused by invalid username/passwords.
From your numbers you have a failure rate of about 5% in our much smaller installation (secondary school) I get up to 30 failed logins per 300 users daily. Typically some users try 10-20 times to login before they realise that they have mispelled their password. I know that because I've been seeing them doing that.
So as long as the same user doesn't try to let's say more than 20 times into the same account per day, I wouldn't worry to much. Unless of couse it is a teacher's or admin's account.
In case you have reason to worry, set an account lockout threshold.
https://docs.moodle.org/400/en/Site_security_settings#:~:text=Account%20lockout%20threshold%3A%20After%20a,an%20unlimited%20number%20of%20times.
So as long as the same user doesn't try to let's say more than 20 times into the same account per day, I wouldn't worry to much. Unless of couse it is a teacher's or admin's account.
In case you have reason to worry, set an account lockout threshold.
https://docs.moodle.org/400/en/Site_security_settings#:~:text=Account%20lockout%20threshold%3A%20After%20a,an%20unlimited%20number%20of%20times.
I'm not talking about the "user/password wrong" type of failed logins. My problem is about invalid login tokens. The login token is a security feature to prevent cross site request forgery (CSRF) on the login form.
Most of the invalid login token errors we saw in the past came from unintended double submissions of the login form. Because, as soon as the first form submission request from the browser reaches the server the used login token (hidden field in the login form) is invalidated (each login token can just be used once and is bind to the session (cookie)). When a second request with the same data reaches the server it logs an invalid login token warning and rejects the login attempt. And though if the first login request succeeded (from the server perspective) the response never reaches the user because the browser aborts that transaction in the moment the form was submitted the second time.
Speaking of login attempts with invalid username/password we see just about 500 per day. That is less than 3%, so I'm not worried about that.
How long does your Moodle take to process a login request - i.e. how long is it from pressing 'log in' until the dashboard appears?
It could be users clicking 'log in' more than once because they think the site is taking too long to log in. This can be caused by login processes that take too long to run (e.g. talking to external systems such as updating enrolments from a database), or certain dashboard blocks that don't scale well and take a long time to run on large sites (I've seen the calendar and upcoming events blocks do this on large sites).
It could be users clicking 'log in' more than once because they think the site is taking too long to log in. This can be caused by login processes that take too long to run (e.g. talking to external systems such as updating enrolments from a database), or certain dashboard blocks that don't scale well and take a long time to run on large sites (I've seen the calendar and upcoming events blocks do this on large sites).
Yes, that is exactly what I tried to say and exactly what happened before (even though the login process takes less than 700ms) we applied the patch from MDL-75083. Since then the number of invalid login token warnings decreased significantly. But now we have still around 1000 (~5,5% of all login attempt) per day (most of them seem to be from Safari browsers, so maybe MDL-75101 plays a role).