CAS Browser Looping

CAS Browser Looping

by Michael Haskell -
Number of replies: 4

In our 2.3.2 installation we periodically see CAS redirect loops that exhaust server resources.  For the most part these loops don't impact performance significantly, however during times of high usage these "extra" requests *do*.

In some instances the loop is compromised of several resource requests looping over and over again and in other cases it is a single looping request.

The loop usually consists of redirects from a request like pluginfile.php/xxxxxx/mod_resource/content then back to the login.php page until the browser is stopped or the user is suspended.

I'm curious if other installations have seen this problem and what solutions they have come up with.  I've been looking into mod_evasive as an option, but I've seen comments about the effectiveness in a load-balanced environment.

I've also seen various issues related to CAS Looping in the tracker:

MDL-21303

MDL-28026

MDL-34533

But nothing about prevention, a fix, or issues related to 2.X.

* Note: right now I'm investigating potentially improper use of CAS's gateway feature in the php code and disabled cookies as possibilities.

Thank you in advance,

Mike

Average of ratings: Useful (1)
In reply to Michael Haskell

Re: CAS Browser Looping

by jason everling -
You are talking about CAS SSO? We have been using CAS for logins for about a year now and haven't had any type of looping. Whats your config like? Do you have all the correct certificates setup in Apache on the moodle server, the CA-Bundle certificate, your intermediary certificate, and finally the server certificate correctly. Did you turn on HTTPS logins only in moodle? All your themes using URLs for linking or path linking? Within CAS you have the correct URL inputed, should resolve like so, https://login.bshp.edu/cas/login?service=https%3A%2F%2Fmoodle2.bshp.edu%2Flogin%2Findex.php We also have turned on the feature to force all users to login and turned use HTTPs for logins in moodle, What do your cas.log and apache logs say?
In reply to Michael Haskell

Re: CAS Browser Looping

by jason everling -
Database or Moodledata sessions? Also another thing that can cause a loop is make sure that the session.save_path in php.ini where the sessions are stored is writable by the web server user. You can run into issues if you are storing the sessions in the database and the sessions folder in the moodeldata directory was never created therefor php cannot successfully save the session information returned by CAS becuase the path is either invalid or not writable.
In reply to jason everling

Re: CAS Browser Looping

by Michael Haskell -

Thanks for your great troubleshooting checklist/questions!  Here's the best information I could procure quickly.

You are talking about CAS SSO?
 - Yes, Yale's CAS for Single Sign-on

We have been using CAS for logins for about a year now and haven't had any type of looping.
 - Interesting, we didn't notice it until we saw slowness and dove into the apache logs.
 - Occasionally we'll see very high usage from a single IP if I run:
    - cat /path/to/your/httpd/access.log | awk '{print $1}' | sort | uniq -c | sort -n
    - grepping the access log on the IP with the most hits (or in the top 5) will sometimes show a series of requests to the pluginfile url mentioned above.  You have to get lucky in terms of day you pick, since the problem only persists until the user closes his or her browser. I'm currently unable to replicate the problem, so I'm guessing here.
   - Could you try looking for these loops if you have a live server?
   - Since posting this issue, we've removed other bottlenecks in our system which have decreased the signficance of this problem.

Whats your config like?
 - A few moodle app servers behind a hardware load-balancer.
 - Multiple moodle versions/instances on each app server
 - CAS and Database on separate servers.

Do you have all the correct certificates setup in Apache on the moodle server
  * the CA-Bundle certificate
  * your intermediary certificate, and
  * finally the server certificate correctly.
    - YES

Did you turn on HTTPS logins only in moodle?
 - No, we require HTTPS for all pages, including login.

All your themes using URLs for linking or path linking?
 - What's the difference between the two?
 - How would this be related to SSO?

Within CAS you have the correct URL inputed, should resolve like so, https://login.bshp.edu/cas/login?service=https%3A%2F%2Fmoodle2.bshp.edu%2Flogin%2Findex.php

We also have turned on the feature to force all users to login and turned use HTTPs for logins in moodle, What do your cas.log and apache logs say?
 - Both

Database or Moodledata sessions?
 - File based sessions (moodledata)

Also another thing that can cause a loop is make sure that the session.save_path in php.ini where the sessions are stored is writable by the web server user.
 - We don't have the session.save_path in php.ini or config.php.  I'm going to rule this out for now, since I can see sessions being written to the appropriate moodledata sessions directory.

You can run into issues if you are storing the sessions in the database and the sessions folder in the moodledata directory was never created therefor php cannot successfully save the session information returned by CAS becuase the path is either invalid or not writable.
 - Interesting, but not applicable with file based sessions.

Note 1: We are running a trimmed down version of the CAS plugin, it has not caused problems in the past. 

Note 2: I can create very small loop if I disable cookies in my browser, but in this case the browser detects the loop and fails gracefully.

Thanks again for the questions.  I'll keep looking.

 

 

 

In reply to Michael Haskell

Re: CAS Browser Looping

by jason everling -

Ah Ok, I checked our access.log and also ssl_access.log and there is nothing really that stands out even with the command you mentioned.

I do however see that most of the pluginfile.php GET requests are coming from the theme files.

We do not use HTTPs for everything, only logins since CAS requires it. Have you tried just using HTTPs from logins only?

We have just started using CAS behind a load balancer and have already started seeing some session issues with a valid session not being found thus making the user login again. Im not sure if its moodle or CAS but we are looking into it, our other applications do not have any session issues.

Which version of CAS? Is it the Jasig built release or a custom version from the original Yale CAS? We use 3.4.11 from the Jasig releases.