Hi!
I was hoping to discuss the way that Moodle authenticates users and recap an event we have recently had in our environment.
Environment: simpleSAML, postgres 9.1, Moodle 2, NLB, several frontends, single database backend
We were initially using a pretty generic configuration with our load balancers: no cookie persistence, least load. We then added simpleSAML. Occasionally, Moodle would go completely unresponsive for several minutes.
Reviewing the logs showed many of the front-ends requesting advisory locks during the creation of the session. These locks would not resolve for quite some time, eventually making it impossible to log in. The issue would resolve itself, but then appear later.
Example log:
< user=user db=db host=172.16.7.67 pid=26285 time=2013-02-04 04:50:21 CST > STATEMENT: SELECT pg_advisory_lock(593819)
< user=user db=db host=172.16.7.45 pid=26382 time=2013-02-04 04:50:32 CST > LOG: process 26382 still waiting for ExclusiveLock on advisory lock [16497,0,593819,1] after 1000.107 ms
< user=user db=db host=172.16.7.45 pid=26382 time=2013-02-04 04:50:32 CST > STATEMENT: SELECT pg_advisory_lock(593819)
< user=user db=db host=172.16.7.66 pid=26615 time=2013-02-04 04:50:34 CST > LOG: process 26615 still waiting for ExclusiveLock on advisory lock [16497,0,593819,1] after 1000.217 ms
After some footwork, it was identified that the interaction between simpleSAML (and I would assume other federated authentication methods) and not using cookie-persistence on the load balancer was generating a "lock storm". It appears (please correct me if I am wrong) that the session data was becoming locked during the initial authentication request and then being locked again when the authentication service returned to the Moodle application.
If I've understood the event correctly, then I'm wondering about the safety of the authentication method when using integrated authentication. I would also like to make note that it should be recommended (if it isn't already) that Moodle instances using a federated login should employ some way to ensure cookie persistence to avoid the locks we encountered.